[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:54 dist-test-slave-npvd krb5kdc[1677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:54 dist-test-slave-npvd krb5kdc[1677](info): set up 2 sockets
Sep 23 22:07:54 dist-test-slave-npvd krb5kdc[1677](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 22:07:55.558981 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.895s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:07:55 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:07:55.660383 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36665
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:55.729408 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:55.729561 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:55.729578 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:55.730535 1690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:55.730569 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:55.730581 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:55.730625 1690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:55.730641 1690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:55.731880 1690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:55.732076 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:55.732271 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:55.736773 1698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:55.736905 1690 server_base.cc:1034] running on GCE node
W20260923 22:07:55.736845 1696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:55.736871 1695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:55.737239 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:55.737481 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:55.738638 1690 hybrid_clock.cc:648] HybridClock initialized: now 1790201275738632 us; error 36 us; skew 500 ppm
Sep 23 22:07:55 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:55.740763 1690 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:07:55.741158 1690 webserver.cc:460] Webserver started at http://127.1.161.190:44431/ using document root <none> and password file <none>
I20260923 22:07:55.741355 1690 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:55.741395 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:55.741520 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:55.742439 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "57ae730548b04e74b0efa07fdc569729"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "6887b34cc68e0928eb59dcb5811d30d7"
server_key_iv: "11f4fe18064c840f5ced50d8df1f40dc"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.742766 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "57ae730548b04e74b0efa07fdc569729"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "6887b34cc68e0928eb59dcb5811d30d7"
server_key_iv: "11f4fe18064c840f5ced50d8df1f40dc"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.744089 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:07:55.744767 1705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:55.744937 1690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:55.745026 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "57ae730548b04e74b0efa07fdc569729"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "6887b34cc68e0928eb59dcb5811d30d7"
server_key_iv: "11f4fe18064c840f5ced50d8df1f40dc"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.745105 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:55.762696 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:55.764223 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:55.764364 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:55.767732 1690 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36665
I20260923 22:07:55.767760 1757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36665 every 8 connection(s)
I20260923 22:07:55.768083 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:55.768857 1758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:55.770757 1758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729: Bootstrap starting.
I20260923 22:07:55.771309 1758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:55.771550 1758 log.cc:824] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:55.772099 1758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729: No bootstrap required, opened a new log
I20260923 22:07:55.773375 1758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } }
I20260923 22:07:55.773478 1758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:55.773518 1758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ae730548b04e74b0efa07fdc569729, State: Initialized, Role: FOLLOWER
I20260923 22:07:55.773646 1758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } }
I20260923 22:07:55.773698 1758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:55.773716 1758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:55.773779 1758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:55.773845 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 1690
I20260923 22:07:55.773948 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:07:55.774307 1758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } }
I20260923 22:07:55.774376 1758 leader_election.cc:302] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57ae730548b04e74b0efa07fdc569729; no voters:
I20260923 22:07:55.774504 1670 external_mini_cluster.cc:1428] Setting key 42ad9966eca42302c173f69fab371afd
I20260923 22:07:55.774542 1758 leader_election.cc:288] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:55.774562 1761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:55.774682 1761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [term 1 LEADER]: Becoming Leader. State: Replica: 57ae730548b04e74b0efa07fdc569729, State: Running, Role: LEADER
I20260923 22:07:55.774765 1758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:55.774749 1761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } }
I20260923 22:07:55.775012 1761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ae730548b04e74b0efa07fdc569729. Latest consensus state: current_term: 1 leader_uuid: "57ae730548b04e74b0efa07fdc569729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } } }
I20260923 22:07:55.775074 1761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:55.775239 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57ae730548b04e74b0efa07fdc569729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae730548b04e74b0efa07fdc569729" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36665 } } }
I20260923 22:07:55.775355 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:55.775784 1766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:55.776221 1766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:55.777379 1766 catalog_manager.cc:1347] Generated new cluster ID: 6f442a6af3c844e0a0794aab5a39f5b6
I20260923 22:07:55.777423 1766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:55 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:55.790689 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:55.775313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.1.161.190:36665)
0923 22:07:55.775487 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:55.775495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:55.775850 (+ 355us) server_negotiation.cc:407] Connection header received
0923 22:07:55.778073 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:55.778080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:55.778128 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:55.778250 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:55.780193 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.781385 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.782714 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.783255 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.783341 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:55.787019 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:55.787023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:55.787024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:55.787034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:55.788906 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:55.789354 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:55.789357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:55.789357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:55.789447 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:55.789682 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:55.789683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:55.789683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:55.789947 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:55.790095 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:55.790363 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 22:07:55.790490 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":137,"thread_start_us":63,"threads_started":1}
I20260923 22:07:55.811478 1766 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:55.811995 1766 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:55.817034 1766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57ae730548b04e74b0efa07fdc569729: Generated new TSK 0
I20260923 22:07:55.817198 1766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:07:55.839437 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:55.913386 1784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:55.913553 1784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:55.913573 1784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:55.914553 1784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:55.914582 1784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:55.914603 1784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:07:55.915974 1784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:55.916132 1784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:55.916296 1784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:55.920971 1789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:55.921013 1790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:55.921047 1784 server_base.cc:1034] running on GCE node
W20260923 22:07:55.920993 1792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:55.921469 1784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:55.921731 1784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:55.922899 1784 hybrid_clock.cc:648] HybridClock initialized: now 1790201275922880 us; error 31 us; skew 500 ppm
Sep 23 22:07:55 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:55.925256 1784 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:07:55.925670 1784 webserver.cc:460] Webserver started at http://127.1.161.129:39111/ using document root <none> and password file <none>
I20260923 22:07:55.925884 1784 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:55.925951 1784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:55.926115 1784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:55.927034 1784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "9f0131a3a7844198a24573f0b9c6bd3b"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "c8d79619c9e98a72180f697effd4c0be"
server_key_iv: "da4ee337b9f3fddee3ec498bcc0242e0"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.927400 1784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "9f0131a3a7844198a24573f0b9c6bd3b"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "c8d79619c9e98a72180f697effd4c0be"
server_key_iv: "da4ee337b9f3fddee3ec498bcc0242e0"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.928589 1784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:55.929253 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:55.929394 1784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:55.929483 1784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "9f0131a3a7844198a24573f0b9c6bd3b"
format_stamp: "Formatted at 2026-09-23 22:07:55 on dist-test-slave-npvd"
server_key: "c8d79619c9e98a72180f697effd4c0be"
server_key_iv: "da4ee337b9f3fddee3ec498bcc0242e0"
server_key_version: "encryptionkey@0"
I20260923 22:07:55.929548 1784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:55.948596 1784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:55.950135 1784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:55.950268 1784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:55.950455 1784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:55.950700 1784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:55.950747 1784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:55.950774 1784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:55.950799 1784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:55.956521 1784 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37621
I20260923 22:07:55.956559 1912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37621 every 8 connection(s)
I20260923 22:07:55.956804 1784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:07:55 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201275, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:55.963042 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 1784
I20260923 22:07:55.963143 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:55.963281 1670 external_mini_cluster.cc:1428] Setting key e2fdbc33e3c3a05832254354d5feea94
I20260923 22:07:55.964618 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:55.957301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:43609 (local address 127.1.161.190:36665)
0923 22:07:55.957360 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:55.957361 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:55.957505 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:07:55.957966 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:55.957967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:55.957977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:55.958028 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:55.958920 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.959443 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.960221 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.960378 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.960422 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:55.962111 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:55.962112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:55.962113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:55.962118 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:55.963412 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:55.963792 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:55.963792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:55.963793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:55.963849 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:55.964100 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:55.964101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:55.964101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:55.964187 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:55.964232 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:55.964507 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 22:07:55.964565 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:07:55.964849 1915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:55.957301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36665 (local address 127.1.161.129:43609)
0923 22:07:55.957420 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:55.957449 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:55.957857 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:55.958122 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:55.958125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:55.958650 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:55.958843 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:55.958847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.959508 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.959508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:55.960158 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:55.960160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:55.960439 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:55.960440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:55.960526 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:55.960866 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:55.960872 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:55.962035 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:55.963489 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:55.963496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:55.963497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:55.963729 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:55.963915 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:55.963916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:55.963916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:55.964041 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:55.964241 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:55.964246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:55.964387 (+ 141us) client_negotiation.cc:769] Sending connection context
0923 22:07:55.964488 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:07:55.964629 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":72,"threads_started":1}
I20260923 22:07:55.965250 1913 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36665
I20260923 22:07:55.965339 1913 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:55.965492 1913 heartbeater.cc:511] Master 127.1.161.190:36665 requested a full tablet report, sending...
I20260923 22:07:55.965894 1722 ts_manager.cc:194] Registered new tserver with Master: 9f0131a3a7844198a24573f0b9c6bd3b (127.1.161.129:37621)
I20260923 22:07:55.966394 1722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:43609
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:07:55.985345 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:56.055321 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:56.055545 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:56.055574 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:56.056692 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:56.056731 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:56.056753 1918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:07:56.057988 1918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:56.058213 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:56.058434 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:56.062624 1924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:56.062670 1926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:56.062629 1923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:56.062788 1918 server_base.cc:1034] running on GCE node
I20260923 22:07:56.062958 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:56.063126 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:56.064297 1918 hybrid_clock.cc:648] HybridClock initialized: now 1790201276064285 us; error 34 us; skew 500 ppm
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:56.066440 1918 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:07:56.066831 1918 webserver.cc:460] Webserver started at http://127.1.161.130:41841/ using document root <none> and password file <none>
I20260923 22:07:56.067027 1918 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:56.067068 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:56.067239 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:56.068182 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "a29ff0506bf343a3b45128f082acbf63"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "e76c99e83792f6722df5d9a69d231af2"
server_key_iv: "9493c4174c991e45c4ff8adafcbdb297"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.068565 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "a29ff0506bf343a3b45128f082acbf63"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "e76c99e83792f6722df5d9a69d231af2"
server_key_iv: "9493c4174c991e45c4ff8adafcbdb297"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.069844 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:56.070554 1933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.070746 1918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:56.070840 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "a29ff0506bf343a3b45128f082acbf63"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "e76c99e83792f6722df5d9a69d231af2"
server_key_iv: "9493c4174c991e45c4ff8adafcbdb297"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.070907 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:56.083571 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:56.085274 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:56.085433 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:56.085637 1918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:56.085934 1918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:56.085964 1918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.085983 1918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:56.086028 1918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.092020 1918 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:46507
I20260923 22:07:56.092059 2046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:46507 every 8 connection(s)
I20260923 22:07:56.092286 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:56.098907 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 1918
I20260923 22:07:56.099004 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:56.099200 1670 external_mini_cluster.cc:1428] Setting key cd46b3c21db8dc5807dff38cb70930d8
I20260923 22:07:56.099581 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.092835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:37925 (local address 127.1.161.190:36665)
0923 22:07:56.092893 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:56.092895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:56.093038 (+ 143us) server_negotiation.cc:407] Connection header received
0923 22:07:56.093369 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:56.093370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:56.093379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:56.093423 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:56.094119 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.094532 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.095318 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.095475 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.095520 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.097216 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:56.097216 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:56.097217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:56.097222 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:56.098286 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.098637 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.098638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.098638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.098686 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.098890 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.098891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.098891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.099028 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:56.099099 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:56.099413 (+ 314us) server_negotiation.cc:299] Negotiation successful
0923 22:07:56.099501 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:07:56.099701 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.092810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36665 (local address 127.1.161.130:37925)
0923 22:07:56.092944 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:56.092971 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:56.093300 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:56.093464 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:56.093467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:56.093901 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:56.094046 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:56.094050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.094596 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.094596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:56.095247 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:56.095249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.095536 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.095536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:56.095616 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.096035 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:56.096048 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:56.097150 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:56.098368 (+ 1218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:56.098373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:56.098375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:56.098575 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:56.098741 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:56.098742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:56.098742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:56.098826 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:56.099089 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:56.099094 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:56.099269 (+ 175us) client_negotiation.cc:769] Sending connection context
0923 22:07:56.099378 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 22:07:56.099516 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260923 22:07:56.100070 2047 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36665
I20260923 22:07:56.100178 2047 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:56.100319 2047 heartbeater.cc:511] Master 127.1.161.190:36665 requested a full tablet report, sending...
I20260923 22:07:56.101035 1722 ts_manager.cc:194] Registered new tserver with Master: a29ff0506bf343a3b45128f082acbf63 (127.1.161.130:46507)
I20260923 22:07:56.101370 1722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:37925
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:07:56.121014 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:56.190155 2052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:56.190332 2052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:56.190354 2052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:56.191367 2052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:56.191396 2052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:56.191419 2052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:07:56.192574 2052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:56.192718 2052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:56.192859 2052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:56.197077 2057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:56.197095 2058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:56.197137 2060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:56.197472 2052 server_base.cc:1034] running on GCE node
I20260923 22:07:56.197649 2052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:56.197821 2052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:56.198931 2052 hybrid_clock.cc:648] HybridClock initialized: now 1790201276198907 us; error 47 us; skew 500 ppm
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:56.200999 2052 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:07:56.201370 2052 webserver.cc:460] Webserver started at http://127.1.161.131:45223/ using document root <none> and password file <none>
I20260923 22:07:56.201552 2052 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:56.201589 2052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:56.201709 2052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:56.202508 2052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "c46f15b1149a4c25810c9fd6fef58740"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "c72191d38bedecaef56c2970a2cfa21d"
server_key_iv: "7d549d6465c8478543205d8f0573f33f"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.202805 2052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "c46f15b1149a4c25810c9fd6fef58740"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "c72191d38bedecaef56c2970a2cfa21d"
server_key_iv: "7d549d6465c8478543205d8f0573f33f"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.203955 2052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:07:56.204526 2067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.204690 2052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:56.204747 2052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "c46f15b1149a4c25810c9fd6fef58740"
format_stamp: "Formatted at 2026-09-23 22:07:56 on dist-test-slave-npvd"
server_key: "c72191d38bedecaef56c2970a2cfa21d"
server_key_iv: "7d549d6465c8478543205d8f0573f33f"
server_key_version: "encryptionkey@0"
I20260923 22:07:56.204851 2052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:56.218133 2052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:56.219766 2052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:56.219899 2052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:56.220077 2052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:56.220321 2052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:56.220369 2052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.220395 2052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:56.220417 2052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:56.225983 2052 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:34223
I20260923 22:07:56.226023 2180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:34223 every 8 connection(s)
I20260923 22:07:56.226308 2052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:56.234297 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.226911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:59335 (local address 127.1.161.190:36665)
0923 22:07:56.227006 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:56.227008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:56.227159 (+ 151us) server_negotiation.cc:407] Connection header received
0923 22:07:56.227623 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:56.227624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:56.227636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:56.227682 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:56.228525 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.229027 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.229904 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.230109 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.230174 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.232067 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:56.232069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:56.232069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:56.232076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:56.233175 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.233504 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.233505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.233505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.233552 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.233761 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.233762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.233763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.233868 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:56.233932 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:56.234146 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 22:07:56.234215 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 22:07:56.234324 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2052
I20260923 22:07:56.234422 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:56.234462 2183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.226842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36665 (local address 127.1.161.131:59335)
0923 22:07:56.227036 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:56.227072 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:56.227520 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:56.227737 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:56.227740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:56.228264 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:56.228433 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:56.228437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.229127 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.229129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:56.229827 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:56.229829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.230189 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.230190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:56.230303 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.230649 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:56.230656 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:56.231978 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:56.233253 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:56.233258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:56.233259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:56.233440 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:56.233622 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:56.233622 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:56.233623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:56.233706 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:56.233924 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:56.233928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:56.234038 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:07:56.234127 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:07:56.234267 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260923 22:07:56.234565 1670 external_mini_cluster.cc:1428] Setting key ed0bbbf9a1c7c684df46035a88e58837
I20260923 22:07:56.235402 2181 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36665
I20260923 22:07:56.235498 2181 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:56.235649 2181 heartbeater.cc:511] Master 127.1.161.190:36665 requested a full tablet report, sending...
I20260923 22:07:56.235960 1722 ts_manager.cc:194] Registered new tserver with Master: c46f15b1149a4c25810c9fd6fef58740 (127.1.161.131:34223)
I20260923 22:07:56.236281 1722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:59335
I20260923 22:07:56.246836 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:07:56.265993 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.259843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.1.161.129:37621)
0923 22:07:56.259979 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:56.259983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:56.260014 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:07:56.260042 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:56.260044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:56.260056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:56.260103 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:56.260532 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.261150 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.261721 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.261989 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.262034 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.263883 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:56.263888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:56.263889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:56.263894 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:56.265045 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.265388 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.265390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.265390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.265446 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.265591 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.265591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.265592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.265692 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:56.265775 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:56.265835 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:07:56.265888 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260923 22:07:56.266353 1827 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54726
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[1677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:07:56.284677 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:56.279219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.1.161.129:37621)
0923 22:07:56.279251 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:56.279252 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:56.279306 (+ 54us) server_negotiation.cc:407] Connection header received
0923 22:07:56.279352 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:56.279353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:56.279360 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:56.279418 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:56.279888 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.280311 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.280928 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:56.281106 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:56.281148 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:56.282839 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:56.282840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:56.282840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:56.282846 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:56.283987 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.284236 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.284236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.284237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.284275 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:56.284394 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:56.284394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:56.284395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:56.284456 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:56.284521 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:56.284579 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:07:56.284628 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:07:56.285111 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 1784
I20260923 22:07:56.289256 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 1918
I20260923 22:07:56.293848 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2052
I20260923 22:07:56.297461 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 1690
2026-09-23T22:07:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1657 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[2195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[2195](info): set up 2 sockets
Sep 23 22:07:56 dist-test-slave-npvd krb5kdc[2195](info): commencing operation
krb5kdc: starting...
W20260923 22:07:57.214113 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.898s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:07:57.314968 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38543
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.386593 2208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.386754 2208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.386771 2208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.387768 2208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:57.387800 2208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.387813 2208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.387825 2208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:57.387835 2208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:57.389158 2208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:57.389333 2208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.389468 2208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.393581 2213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.393570 2214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.393570 2216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.393704 2208 server_base.cc:1034] running on GCE node
I20260923 22:07:57.393968 2208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.394191 2208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.395331 2208 hybrid_clock.cc:648] HybridClock initialized: now 1790201277395318 us; error 28 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.397330 2208 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:07:57.397714 2208 webserver.cc:460] Webserver started at http://127.1.161.190:39877/ using document root <none> and password file <none>
I20260923 22:07:57.397897 2208 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.397953 2208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.398062 2208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.399016 2208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "a00d063dade3467b931851ee98dc3649"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "927a09166de3a21f9a649a35f70b9f23"
server_key_iv: "35ce9a207b0a1808b4d10bc7ef5f66ae"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.399400 2208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "a00d063dade3467b931851ee98dc3649"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "927a09166de3a21f9a649a35f70b9f23"
server_key_iv: "35ce9a207b0a1808b4d10bc7ef5f66ae"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.400506 2208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:57.401068 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.401209 2208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:57.401289 2208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "a00d063dade3467b931851ee98dc3649"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "927a09166de3a21f9a649a35f70b9f23"
server_key_iv: "35ce9a207b0a1808b4d10bc7ef5f66ae"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.401364 2208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.416595 2208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.418027 2208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.418212 2208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.421720 2208 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38543
I20260923 22:07:57.421743 2275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38543 every 8 connection(s)
I20260923 22:07:57.422025 2208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:57.422715 2276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:57.424703 2276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649: Bootstrap starting.
I20260923 22:07:57.425213 2276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:57.425472 2276 log.cc:824] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:57.426034 2276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649: No bootstrap required, opened a new log
I20260923 22:07:57.427388 2276 raft_consensus.cc:348] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } }
I20260923 22:07:57.427520 2276 raft_consensus.cc:374] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:57.427541 2276 raft_consensus.cc:729] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00d063dade3467b931851ee98dc3649, State: Initialized, Role: FOLLOWER
I20260923 22:07:57.427695 2276 consensus_queue.cc:260] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } }
I20260923 22:07:57.427752 2276 raft_consensus.cc:388] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:57.427788 2276 raft_consensus.cc:482] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:57.427840 2276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:57.428365 2276 raft_consensus.cc:504] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } }
I20260923 22:07:57.428465 2276 leader_election.cc:302] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a00d063dade3467b931851ee98dc3649; no voters:
I20260923 22:07:57.428651 2276 leader_election.cc:288] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:57.428654 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2208
I20260923 22:07:57.428671 2279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:57.428740 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:07:57.428781 2279 raft_consensus.cc:686] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [term 1 LEADER]: Becoming Leader. State: Replica: a00d063dade3467b931851ee98dc3649, State: Running, Role: LEADER
I20260923 22:07:57.428848 2279 consensus_queue.cc:237] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } }
I20260923 22:07:57.428896 1670 external_mini_cluster.cc:1428] Setting key b850233c47c98835b04eb01fdd21b509
I20260923 22:07:57.428900 2276 sys_catalog.cc:564] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:57.429276 2280 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a00d063dade3467b931851ee98dc3649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } } }
I20260923 22:07:57.429549 2280 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:57.429477 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a00d063dade3467b931851ee98dc3649. Latest consensus state: current_term: 1 leader_uuid: "a00d063dade3467b931851ee98dc3649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00d063dade3467b931851ee98dc3649" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38543 } } }
I20260923 22:07:57.429704 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649 [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:57.429991 2290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:57.430476 2290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:57.432753 2290 catalog_manager.cc:1347] Generated new cluster ID: 87cfaf7c99bb47a782691f1cc24783c6
I20260923 22:07:57.432796 2290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:57.439533 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.429270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.1.161.190:38543)
0923 22:07:57.429998 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.430005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.430047 (+ 42us) server_negotiation.cc:407] Connection header received
0923 22:07:57.430144 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.430149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.430196 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.430352 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.431351 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.432157 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.433108 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.433591 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.433672 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.436069 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.436073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.436073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.436085 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.437902 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.438289 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.438292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.438292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.438420 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.438611 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.438612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.438612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.438906 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.439051 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.439150 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.439327 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":82,"threads_started":1}
I20260923 22:07:57.443544 2290 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:57.443975 2290 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:57.450713 2290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a00d063dade3467b931851ee98dc3649: Generated new TSK 0
I20260923 22:07:57.450853 2290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:07:57.480077 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.550220 2302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.550390 2302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.550434 2302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.551424 2302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.551458 2302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.551482 2302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:07:57.552662 2302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:57.552814 2302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.553026 2302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.557328 2308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.557351 2307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.557386 2310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.557392 2302 server_base.cc:1034] running on GCE node
I20260923 22:07:57.557628 2302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.557790 2302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.558917 2302 hybrid_clock.cc:648] HybridClock initialized: now 1790201277558907 us; error 24 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.561080 2302 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:07:57.561457 2302 webserver.cc:460] Webserver started at http://127.1.161.129:41147/ using document root <none> and password file <none>
I20260923 22:07:57.561648 2302 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.561705 2302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.561831 2302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.562629 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "a51e8d7596dc47deacb72f1a6d9a8869"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "b3cd6e361469978cb1d054702cb2060c"
server_key_iv: "e02b72b90c1019367d1506f765b3d9d2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.562923 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "a51e8d7596dc47deacb72f1a6d9a8869"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "b3cd6e361469978cb1d054702cb2060c"
server_key_iv: "e02b72b90c1019367d1506f765b3d9d2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.564147 2302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:57.564805 2317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.564975 2302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:57.565068 2302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "a51e8d7596dc47deacb72f1a6d9a8869"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "b3cd6e361469978cb1d054702cb2060c"
server_key_iv: "e02b72b90c1019367d1506f765b3d9d2"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.565155 2302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.585471 2302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.587491 2302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.587622 2302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.587800 2302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:57.588053 2302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:57.588100 2302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.588173 2302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:57.588201 2302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.593513 2302 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:34923
I20260923 22:07:57.593561 2430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:34923 every 8 connection(s)
I20260923 22:07:57.593786 2302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:57.601648 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.594392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:43665 (local address 127.1.161.190:38543)
0923 22:07:57.594430 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.594432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.594587 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:07:57.595418 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.595419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.595429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.595469 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.595943 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.596358 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.597177 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.597347 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.597392 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.599352 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.599353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.599353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.599364 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.600643 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.600969 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.600969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.600970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.601012 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.601219 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.601219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.601219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.601301 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.601342 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.601539 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.601596 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:07:57.601814 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.594303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38543 (local address 127.1.161.129:43665)
0923 22:07:57.594485 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.594512 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.595332 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.595521 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.595524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.595727 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:57.595879 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.595882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.596437 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.596437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.597107 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.597109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.597421 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.597422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.597511 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.597865 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:57.597872 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:57.599220 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:57.600712 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.600718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.600719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.600906 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.601076 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.601077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.601077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.601160 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.601351 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:57.601355 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:57.601444 (+ 89us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.601521 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.601617 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260923 22:07:57.602305 2431 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38543
I20260923 22:07:57.602396 2431 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:57.602545 2431 heartbeater.cc:511] Master 127.1.161.190:38543 requested a full tablet report, sending...
I20260923 22:07:57.603042 2240 ts_manager.cc:194] Registered new tserver with Master: a51e8d7596dc47deacb72f1a6d9a8869 (127.1.161.129:34923)
I20260923 22:07:57.603603 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2302
I20260923 22:07:57.603686 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:57.603818 2240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:43665
I20260923 22:07:57.603840 1670 external_mini_cluster.cc:1428] Setting key 99e7441c3e43bda69bfa7e5a06982c26
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:07:57.625309 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.696074 2436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.696225 2436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.696244 2436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.697273 2436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.697309 2436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.697333 2436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:07:57.698770 2436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:57.698990 2436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.699242 2436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.703389 2441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.703395 2442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.703395 2444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.703488 2436 server_base.cc:1034] running on GCE node
I20260923 22:07:57.703768 2436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.703923 2436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.705040 2436 hybrid_clock.cc:648] HybridClock initialized: now 1790201277705033 us; error 23 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.707027 2436 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:07:57.707425 2436 webserver.cc:460] Webserver started at http://127.1.161.130:33931/ using document root <none> and password file <none>
I20260923 22:07:57.707605 2436 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.707660 2436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.707772 2436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.708675 2436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "d5f91042a853416688642c77c25f61e1"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "05e8ba3355708cb085d4ac04ce5f7f53"
server_key_iv: "d6f6cc948175e92c0bbabc0797281a83"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.709008 2436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "d5f91042a853416688642c77c25f61e1"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "05e8ba3355708cb085d4ac04ce5f7f53"
server_key_iv: "d6f6cc948175e92c0bbabc0797281a83"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.710132 2436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:57.710727 2451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.710878 2436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:57.710956 2436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "d5f91042a853416688642c77c25f61e1"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "05e8ba3355708cb085d4ac04ce5f7f53"
server_key_iv: "d6f6cc948175e92c0bbabc0797281a83"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.711027 2436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.719278 2436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.720708 2436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.720837 2436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.721002 2436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:57.721243 2436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:57.721288 2436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.721329 2436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:57.721354 2436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.726516 2436 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:36225
I20260923 22:07:57.726572 2564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:36225 every 8 connection(s)
I20260923 22:07:57.726805 2436 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:57.728776 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2436
I20260923 22:07:57.728856 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:57.729024 1670 external_mini_cluster.cc:1428] Setting key 2fc290197f5aa69aaffe862ee4755579
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:57.735311 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.727370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:52909 (local address 127.1.161.190:38543)
0923 22:07:57.727445 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.727447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.727571 (+ 124us) server_negotiation.cc:407] Connection header received
0923 22:07:57.728298 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.728299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.728308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.728358 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.728988 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.729748 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.730623 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.730903 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.730940 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.732730 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.732731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.732731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.732739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.734107 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.734453 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.734454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.734455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.734509 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.734717 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.734718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.734718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.734836 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.734903 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.735100 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.735222 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:07:57.735335 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.727333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38543 (local address 127.1.161.130:52909)
0923 22:07:57.727475 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.727501 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.728219 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.728401 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.728404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.728675 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:57.728876 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.728881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.729820 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.729821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.730486 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.730488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.731015 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.731015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.731139 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.731530 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:57.731537 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:57.732650 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:57.734176 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.734181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.734182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.734369 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.734572 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.734573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.734573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.734657 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.734897 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:57.734900 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:57.734987 (+ 87us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.735055 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.735157 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260923 22:07:57.735729 2565 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38543
I20260923 22:07:57.735807 2565 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:57.735953 2565 heartbeater.cc:511] Master 127.1.161.190:38543 requested a full tablet report, sending...
I20260923 22:07:57.736246 2240 ts_manager.cc:194] Registered new tserver with Master: d5f91042a853416688642c77c25f61e1 (127.1.161.130:36225)
I20260923 22:07:57.736557 2240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:52909
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:07:57.751143 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:57.821112 2570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:57.821287 2570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:57.821318 2570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:57.822332 2570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:57.822389 2570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:57.822428 2570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:07:57.823697 2570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:57.823915 2570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:57.824092 2570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:57.828311 2575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:57.828565 2576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.828608 2570 server_base.cc:1034] running on GCE node
W20260923 22:07:57.828610 2578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:57.828966 2570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:57.829193 2570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:57.830340 2570 hybrid_clock.cc:648] HybridClock initialized: now 1790201277830335 us; error 37 us; skew 500 ppm
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:57.832525 2570 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:07:57.832818 2570 webserver.cc:460] Webserver started at http://127.1.161.131:38235/ using document root <none> and password file <none>
I20260923 22:07:57.832978 2570 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:57.833012 2570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:57.833081 2570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:57.833832 2570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "df242c09fd3342ec8e1f98da57425d9a"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "4c0cfdca948be15afee4b330d7dcb838"
server_key_iv: "751d81d723a06972b576ab0f85bf685a"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.834081 2570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "df242c09fd3342ec8e1f98da57425d9a"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "4c0cfdca948be15afee4b330d7dcb838"
server_key_iv: "751d81d723a06972b576ab0f85bf685a"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.835088 2570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:57.835667 2585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.835793 2570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.835860 2570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "df242c09fd3342ec8e1f98da57425d9a"
format_stamp: "Formatted at 2026-09-23 22:07:57 on dist-test-slave-npvd"
server_key: "4c0cfdca948be15afee4b330d7dcb838"
server_key_iv: "751d81d723a06972b576ab0f85bf685a"
server_key_version: "encryptionkey@0"
I20260923 22:07:57.835901 2570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:57.855540 2570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:57.856877 2570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:57.856961 2570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:57.857086 2570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:07:57.857324 2570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:57.857352 2570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.857368 2570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:57.857409 2570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:57.862285 2570 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:44707
I20260923 22:07:57.862396 2698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:44707 every 8 connection(s)
I20260923 22:07:57.862641 2570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:57.864776 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2570
I20260923 22:07:57.864895 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:57.865027 1670 external_mini_cluster.cc:1428] Setting key 6626d7e0bea1cb70d4ce991afdf69212
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:57.870910 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.863268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:55547 (local address 127.1.161.190:38543)
0923 22:07:57.863318 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.863320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.863501 (+ 181us) server_negotiation.cc:407] Connection header received
0923 22:07:57.864266 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.864267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.864276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.864342 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.864861 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.865468 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.866394 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.866547 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.866588 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.868370 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.868372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.868372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.868379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.869704 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.870094 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.870095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.870096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.870150 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.870381 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.870382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.870383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.870477 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.870541 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.870765 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.870836 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:07:57.871034 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.863252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38543 (local address 127.1.161.131:55547)
0923 22:07:57.863382 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.863416 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.864194 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.864387 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.864390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.864588 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:57.864752 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.864756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.865551 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.865552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.866327 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.866329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.866593 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.866594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.866666 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.866970 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:57.866993 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:57.868273 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:57.869790 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.869795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.869797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.869985 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.870210 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:57.870210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:57.870211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:57.870307 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:57.870533 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:57.870538 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:57.870652 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.870735 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.870862 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260923 22:07:57.871423 2699 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38543
I20260923 22:07:57.871496 2699 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:57.871601 2699 heartbeater.cc:511] Master 127.1.161.190:38543 requested a full tablet report, sending...
I20260923 22:07:57.871855 2240 ts_manager.cc:194] Registered new tserver with Master: df242c09fd3342ec8e1f98da57425d9a (127.1.161.131:44707)
I20260923 22:07:57.872118 2240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:55547
I20260923 22:07:57.876398 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:07:57.882476 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.877667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.1.161.190:38543)
0923 22:07:57.877709 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.877711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.877806 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:07:57.877838 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.877839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.877845 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.877887 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.878339 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.878759 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.879467 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.879607 (+ 140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.879649 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.880532 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.880533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.880533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.880538 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.881705 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.881959 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.881959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.881959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.881999 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.882100 (+ 101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.882100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.882100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.882195 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.882265 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.882344 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.882413 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:07:57.884099 2240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:57.885401 2240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:57.890936 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.1.161.131:44707)
0923 22:07:57.887335 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.887338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.887658 (+ 320us) server_negotiation.cc:407] Connection header received
0923 22:07:57.887735 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.887736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.887749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.887820 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:57.888004 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.889103 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.889958 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.890590 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.890670 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.890716 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.890838 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.890874 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
I20260923 22:07:57.890931 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:44707 (local address 127.0.0.1:50312)
0923 22:07:57.887564 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.887595 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.887660 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.887886 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.887887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.887889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:57.887933 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.887935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.889205 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.889205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.889883 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.889885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.890671 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.890671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.890746 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.890757 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.890822 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.890867 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":22,"thread_start_us":39,"threads_started":1}
I20260923 22:07:57.891381 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:34923 (local address 127.0.0.1:45596)
0923 22:07:57.887748 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.887754 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.887790 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.887998 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.887998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.888000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:57.888024 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.888025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.888994 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.888995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.890277 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.890279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.891138 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.891139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.891212 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.891216 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.891265 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.891305 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":22,"thread_start_us":54,"threads_started":1}
I20260923 22:07:57.891566 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.1.161.130:36225)
0923 22:07:57.887541 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.887544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.887655 (+ 111us) server_negotiation.cc:407] Connection header received
0923 22:07:57.887736 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.887737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.887749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.887821 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:57.887987 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.889041 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.890261 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.890975 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.891043 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.891094 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.891419 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.891466 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260923 22:07:57.891381 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36225 (local address 127.0.0.1:34670)
0923 22:07:57.887565 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.887595 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.887660 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.887867 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.887869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.887871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:57.887925 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.887927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.889135 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.889136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.890182 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.890185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.891128 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.891129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.891212 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.891216 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.891265 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.891312 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":7,"thread_start_us":72,"threads_started":1}
I20260923 22:07:57.891674 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.887319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.1.161.129:34923)
0923 22:07:57.887453 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.887456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.887793 (+ 337us) server_negotiation.cc:407] Connection header received
0923 22:07:57.887853 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.887856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.887869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.887928 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:57.888070 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.888895 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.890367 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.891017 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.891069 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.891112 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.891554 (+ 442us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.891596 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260923 22:07:57.891759 2633 tablet_service.cc:1467] Processing CreateTablet for tablet d7bfbd3c72fe458f86ef557432298cce (DEFAULT_TABLE table=test-table [id=3c300e660aae44879ecbae93499d4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:57.892043 2633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7bfbd3c72fe458f86ef557432298cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:57.892122 2365 tablet_service.cc:1467] Processing CreateTablet for tablet d7bfbd3c72fe458f86ef557432298cce (DEFAULT_TABLE table=test-table [id=3c300e660aae44879ecbae93499d4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:57.892318 2365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7bfbd3c72fe458f86ef557432298cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:57.894526 2713 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: Bootstrap starting.
I20260923 22:07:57.894999 2713 tablet_bootstrap.cc:654] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:57.895402 2714 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: Bootstrap starting.
I20260923 22:07:57.895485 2713 log.cc:824] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:57.895913 2714 tablet_bootstrap.cc:654] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:57.896358 2714 log.cc:824] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:57.896464 2499 tablet_service.cc:1467] Processing CreateTablet for tablet d7bfbd3c72fe458f86ef557432298cce (DEFAULT_TABLE table=test-table [id=3c300e660aae44879ecbae93499d4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:57.896728 2499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7bfbd3c72fe458f86ef557432298cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:57.897032 2714 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: No bootstrap required, opened a new log
I20260923 22:07:57.897094 2714 ts_tablet_manager.cc:1397] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:57.898764 2714 raft_consensus.cc:348] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.898936 2714 raft_consensus.cc:374] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:57.898969 2714 raft_consensus.cc:729] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df242c09fd3342ec8e1f98da57425d9a, State: Initialized, Role: FOLLOWER
I20260923 22:07:57.898981 2713 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: No bootstrap required, opened a new log
I20260923 22:07:57.899032 2713 ts_tablet_manager.cc:1397] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 22:07:57.899096 2714 consensus_queue.cc:260] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.899399 2714 ts_tablet_manager.cc:1428] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:07:57.899521 2699 heartbeater.cc:503] Master 127.1.161.190:38543 was elected leader, sending a full tablet report...
I20260923 22:07:57.899896 2718 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Bootstrap starting.
I20260923 22:07:57.900827 2718 tablet_bootstrap.cc:654] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:57.901274 2718 log.cc:824] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:57.901875 2718 tablet_bootstrap.cc:492] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: No bootstrap required, opened a new log
I20260923 22:07:57.901932 2718 ts_tablet_manager.cc:1397] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:57.902026 2713 raft_consensus.cc:348] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.902161 2713 raft_consensus.cc:374] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:57.902189 2713 raft_consensus.cc:729] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51e8d7596dc47deacb72f1a6d9a8869, State: Initialized, Role: FOLLOWER
I20260923 22:07:57.902283 2713 consensus_queue.cc:260] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.902499 2713 ts_tablet_manager.cc:1428] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:07:57.902521 2431 heartbeater.cc:503] Master 127.1.161.190:38543 was elected leader, sending a full tablet report...
I20260923 22:07:57.903558 2718 raft_consensus.cc:348] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.903659 2718 raft_consensus.cc:374] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:57.903677 2718 raft_consensus.cc:729] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f91042a853416688642c77c25f61e1, State: Initialized, Role: FOLLOWER
I20260923 22:07:57.903749 2718 consensus_queue.cc:260] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.903903 2718 ts_tablet_manager.cc:1428] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:57.903967 2565 heartbeater.cc:503] Master 127.1.161.190:38543 was elected leader, sending a full tablet report...
I20260923 22:07:57.946620 2720 raft_consensus.cc:482] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:57.946691 2720 raft_consensus.cc:504] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.946986 2720 leader_election.cc:288] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5f91042a853416688642c77c25f61e1 (127.1.161.130:36225), df242c09fd3342ec8e1f98da57425d9a (127.1.161.131:44707)
I20260923 22:07:57.949846 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.947211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:35037 (local address 127.1.161.131:44707)
0923 22:07:57.947256 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.947258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.947266 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:57.947348 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.947349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.947368 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.947417 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:57.947564 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.947982 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.948997 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.949548 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.949595 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.949600 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.949758 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.949790 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:07:57.949877 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.947266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:56039 (local address 127.1.161.130:36225)
0923 22:07:57.947299 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.947300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.947423 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:07:57.947446 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.947446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.947455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.947517 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:57.947650 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.948060 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.948997 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.949590 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.949658 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.949665 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.949790 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.949820 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:07:57.949891 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.947154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:44707 (local address 127.1.161.129:35037)
0923 22:07:57.947223 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.947233 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.947291 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.947438 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.947440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.947441 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:57.947501 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.947502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.948053 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.948053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.948901 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.948903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.949619 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.949620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.949691 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.949694 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.949743 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.949802 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":34}
I20260923 22:07:57.950081 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.947154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36225 (local address 127.1.161.129:56039)
0923 22:07:57.947333 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:57.947342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:57.947400 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:57.947517 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:57.947518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:57.947519 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:57.947589 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.947590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.948140 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.948141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.948910 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:57.948912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.949665 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.949665 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:57.949709 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.949712 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:07:57.949773 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:07:57.949804 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":45,"thread_start_us":76,"threads_started":1}
I20260923 22:07:57.950240 2653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7bfbd3c72fe458f86ef557432298cce" candidate_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df242c09fd3342ec8e1f98da57425d9a" is_pre_election: true
I20260923 22:07:57.950376 2653 raft_consensus.cc:2454] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a51e8d7596dc47deacb72f1a6d9a8869 in term 0.
I20260923 22:07:57.950410 2519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7bfbd3c72fe458f86ef557432298cce" candidate_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f91042a853416688642c77c25f61e1" is_pre_election: true
I20260923 22:07:57.950521 2519 raft_consensus.cc:2454] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a51e8d7596dc47deacb72f1a6d9a8869 in term 0.
I20260923 22:07:57.950562 2320 leader_election.cc:302] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a51e8d7596dc47deacb72f1a6d9a8869, df242c09fd3342ec8e1f98da57425d9a; no voters:
I20260923 22:07:57.950682 2720 raft_consensus.cc:2781] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:57.950738 2720 raft_consensus.cc:482] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:57.950753 2720 raft_consensus.cc:3037] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:57.951406 2720 raft_consensus.cc:504] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.951516 2720 leader_election.cc:288] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [CANDIDATE]: Term 1 election: Requested vote from peers d5f91042a853416688642c77c25f61e1 (127.1.161.130:36225), df242c09fd3342ec8e1f98da57425d9a (127.1.161.131:44707)
I20260923 22:07:57.951617 2519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7bfbd3c72fe458f86ef557432298cce" candidate_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f91042a853416688642c77c25f61e1"
I20260923 22:07:57.951671 2519 raft_consensus.cc:3037] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:57.951699 2653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7bfbd3c72fe458f86ef557432298cce" candidate_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df242c09fd3342ec8e1f98da57425d9a"
I20260923 22:07:57.951771 2653 raft_consensus.cc:3037] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:57.952203 2519 raft_consensus.cc:2454] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a51e8d7596dc47deacb72f1a6d9a8869 in term 1.
I20260923 22:07:57.952364 2319 leader_election.cc:302] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a51e8d7596dc47deacb72f1a6d9a8869, d5f91042a853416688642c77c25f61e1; no voters:
I20260923 22:07:57.952450 2720 raft_consensus.cc:2781] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:57.952447 2653 raft_consensus.cc:2454] T d7bfbd3c72fe458f86ef557432298cce P df242c09fd3342ec8e1f98da57425d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a51e8d7596dc47deacb72f1a6d9a8869 in term 1.
I20260923 22:07:57.952576 2720 raft_consensus.cc:686] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [term 1 LEADER]: Becoming Leader. State: Replica: a51e8d7596dc47deacb72f1a6d9a8869, State: Running, Role: LEADER
I20260923 22:07:57.952648 2720 consensus_queue.cc:237] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } }
I20260923 22:07:57.953356 2239 catalog_manager.cc:5495] T d7bfbd3c72fe458f86ef557432298cce P a51e8d7596dc47deacb72f1a6d9a8869 reported cstate change: term changed from 0 to 1, leader changed from <none> to a51e8d7596dc47deacb72f1a6d9a8869 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f91042a853416688642c77c25f61e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df242c09fd3342ec8e1f98da57425d9a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a51e8d7596dc47deacb72f1a6d9a8869" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 34923 } health_report { overall_health: HEALTHY } } }
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:07:57.971956 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.964945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.1.161.129:34923)
0923 22:07:57.965009 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.965011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.965082 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:07:57.965133 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.965133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.965142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.965184 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.965670 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.966077 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.966723 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.966952 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.966992 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.969386 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.969393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.969393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.969399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.970852 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.971213 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.971217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.971217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.971283 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.971461 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.971462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.971463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.971597 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.971757 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.971795 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.971862 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260923 22:07:57.977772 2566 tablet.cc:2369] T d7bfbd3c72fe458f86ef557432298cce P d5f91042a853416688642c77c25f61e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:07:57.991884 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:57.986036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.1.161.129:34923)
0923 22:07:57.986092 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:57.986094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:57.986147 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:07:57.986218 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:57.986219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:57.986228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:57.986273 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:57.986688 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.987070 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.987713 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:57.987896 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:57.987959 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:57.989850 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:57.989850 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:57.989851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:57.989857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:57.991108 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.991410 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.991411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.991411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.991451 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:57.991581 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:57.991581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:57.991581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:57.991643 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:57.991698 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:57.991791 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:07:57.991835 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 23 22:07:57 dist-test-slave-npvd krb5kdc[2195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:07:58 dist-test-slave-npvd krb5kdc[2195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:07:58.010243 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:58.004398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.1.161.129:34923)
0923 22:07:58.004443 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:58.004444 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:58.004519 (+ 75us) server_negotiation.cc:407] Connection header received
0923 22:07:58.004553 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:58.004554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:58.004568 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:58.004604 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:58.005000 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.005390 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.006025 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:58.006207 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:58.006260 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:58.008152 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:58.008153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:58.008153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:58.008158 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:58.009534 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.009779 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.009780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.009780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.009818 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:58.009937 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:58.009937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:58.009938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:58.009997 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:58.010060 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:58.010151 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:07:58.010197 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260923 22:07:58.010602 2345 tablet_service.cc:2908] Rejecting scan request for tablet d7bfbd3c72fe458f86ef557432298cce: Uninitialized: clean time has not yet been initialized
I20260923 22:07:58.011356 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2302
I20260923 22:07:58.016005 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2436
I20260923 22:07:58.020117 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2570
I20260923 22:07:58.024247 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2208
2026-09-23T22:07:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1727 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:07:58 dist-test-slave-npvd krb5kdc[2737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:07:58 dist-test-slave-npvd krb5kdc[2737](info): set up 2 sockets
Sep 23 22:07:58 dist-test-slave-npvd krb5kdc[2737](info): commencing operation
krb5kdc: starting...
W20260923 22:07:58.946224 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.902s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:07:59.043330 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38717
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.111426 2750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.111600 2750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.111630 2750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.112596 2750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:07:59.112650 2750 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:07:59.112669 2750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.112686 2750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.112713 2750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:07:59.112733 2750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:07:59.113942 2750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:59.114161 2750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.114342 2750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.118368 2756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.118402 2758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.118439 2750 server_base.cc:1034] running on GCE node
W20260923 22:07:59.118412 2755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.118762 2750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.119048 2750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.120204 2750 hybrid_clock.cc:648] HybridClock initialized: now 1790201279120185 us; error 34 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.126806 2750 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:07:59.127244 2750 webserver.cc:460] Webserver started at http://127.1.161.190:45235/ using document root <none> and password file <none>
I20260923 22:07:59.127439 2750 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.127496 2750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.127609 2750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.128456 2750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "72a09490abc946ba867acfadbafbf1ea"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "2636faf78d6297e4948fd3093427929e"
server_key_iv: "789d6bb5c357c7c355e2fcae19770651"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.128788 2750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "72a09490abc946ba867acfadbafbf1ea"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "2636faf78d6297e4948fd3093427929e"
server_key_iv: "789d6bb5c357c7c355e2fcae19770651"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.129971 2750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:07:59.130628 2765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.130789 2750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:59.130878 2750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "72a09490abc946ba867acfadbafbf1ea"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "2636faf78d6297e4948fd3093427929e"
server_key_iv: "789d6bb5c357c7c355e2fcae19770651"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.130954 2750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.142483 2750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.143983 2750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.144116 2750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.148128 2750 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38717
I20260923 22:07:59.148173 2827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38717 every 8 connection(s)
I20260923 22:07:59.148473 2750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:07:59.149060 2828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.150876 2828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Bootstrap starting.
I20260923 22:07:59.151384 2828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.151590 2828 log.cc:824] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.152096 2828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: No bootstrap required, opened a new log
I20260923 22:07:59.153751 2828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } }
I20260923 22:07:59.153862 2828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.153882 2828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a09490abc946ba867acfadbafbf1ea, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.153962 2828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } }
I20260923 22:07:59.154012 2828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:07:59.154030 2828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:07:59.154052 2828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:59.154443 2828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } }
I20260923 22:07:59.154510 2828 leader_election.cc:302] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a09490abc946ba867acfadbafbf1ea; no voters:
I20260923 22:07:59.154610 2828 leader_election.cc:288] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:07:59.154672 2831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:59.154814 2831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [term 1 LEADER]: Becoming Leader. State: Replica: 72a09490abc946ba867acfadbafbf1ea, State: Running, Role: LEADER
I20260923 22:07:59.154815 2828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:07:59.154896 2831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } }
I20260923 22:07:59.155236 2832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a09490abc946ba867acfadbafbf1ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } } }
I20260923 22:07:59.155367 2833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a09490abc946ba867acfadbafbf1ea. Latest consensus state: current_term: 1 leader_uuid: "72a09490abc946ba867acfadbafbf1ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a09490abc946ba867acfadbafbf1ea" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38717 } } }
I20260923 22:07:59.155465 2833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:59.155445 2832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea [sys.catalog]: This master's current role is: LEADER
I20260923 22:07:59.155766 2841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:07:59.156164 2841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:07:59.156816 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2750
I20260923 22:07:59.156905 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:07:59.157065 1670 external_mini_cluster.cc:1428] Setting key 0c1cd0dda748bdcebea5f9231e0db8b4
I20260923 22:07:59.157574 2841 catalog_manager.cc:1347] Generated new cluster ID: ad1cbea4daa7496189108ce36229896a
I20260923 22:07:59.157620 2841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:59.166955 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.157502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.1.161.190:38717)
0923 22:07:59.159279 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.159286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.159310 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:07:59.159391 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.159396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.159441 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.159591 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.160122 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.161180 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.161846 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.162211 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.162320 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.164296 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.164299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.164299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.164314 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.165667 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.165987 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.165990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.165990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.166070 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.166251 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.166252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.166252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.166440 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.166585 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.166663 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.166779 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":67,"threads_started":1}
I20260923 22:07:59.172179 2841 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:07:59.172819 2841 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:07:59.182802 2841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Generated new TSK 0
I20260923 22:07:59.182940 2841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:07:59.203835 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38717
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.272715 2854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.272871 2854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.272889 2854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.273865 2854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.273900 2854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.273916 2854 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:59.273969 2854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:07:59.275401 2854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:38717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:59.275614 2854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.275870 2854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.280411 2859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.280418 2860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.280542 2854 server_base.cc:1034] running on GCE node
W20260923 22:07:59.280421 2862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.280761 2854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.280959 2854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.282104 2854 hybrid_clock.cc:648] HybridClock initialized: now 1790201279282083 us; error 30 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.284139 2854 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:07:59.284545 2854 webserver.cc:460] Webserver started at http://127.1.161.129:43629/ using document root <none> and password file <none>
I20260923 22:07:59.284749 2854 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.284812 2854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.284956 2854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.285844 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "26a19aac4fe4f80f523d0983d8f429fa"
server_key_iv: "82e31e9975a7615dcdb714f368e33130"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.286136 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "26a19aac4fe4f80f523d0983d8f429fa"
server_key_iv: "82e31e9975a7615dcdb714f368e33130"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.287289 2854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:07:59.287901 2869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.288051 2854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:59.288130 2854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "26a19aac4fe4f80f523d0983d8f429fa"
server_key_iv: "82e31e9975a7615dcdb714f368e33130"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.288201 2854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.298123 2854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.299774 2854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.299901 2854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.300271 2854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.300325 2854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.300376 2854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.300419 2854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:59.306432 2854 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46123
I20260923 22:07:59.306490 2984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46123 every 8 connection(s)
I20260923 22:07:59.306723 2854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:07:59.307374 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2854
I20260923 22:07:59.307473 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:07:59.307653 1670 external_mini_cluster.cc:1428] Setting key 0c8bb08665ced225781723a9f2de03d0
I20260923 22:07:59.309080 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.300686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.129:58505)
0923 22:07:59.300821 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.300853 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.301211 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.301426 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.301429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.301738 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.301962 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.301968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.302850 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.302851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.303738 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.303741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.304258 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.304259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.304367 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.304777 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.304784 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.306018 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.307863 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.307868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.307870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.308081 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.308300 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.308301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.308301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.308405 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.308679 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.308682 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.308773 (+ 91us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.308844 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.308947 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260923 22:07:59.309124 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.300709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:58505 (local address 127.1.161.190:38717)
0923 22:07:59.300768 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.300771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.300928 (+ 157us) server_negotiation.cc:407] Connection header received
0923 22:07:59.301302 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.301303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.301318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.301389 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.302056 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.302758 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.303832 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.304149 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.304228 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.306084 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.306086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.306086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.306095 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.307776 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.308160 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.308162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.308162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.308243 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.308475 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.308476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.308477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.308614 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.308692 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.308890 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.308979 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:07:59.310593 2985 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38717
I20260923 22:07:59.310657 2985 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.310786 2985 heartbeater.cc:511] Master 127.1.161.190:38717 requested a full tablet report, sending...
I20260923 22:07:59.311236 2781 ts_manager.cc:194] Registered new tserver with Master: 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123)
I20260923 22:07:59.311808 2781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:58505
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
I20260923 22:07:59.315809 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.310816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:52945 (local address 127.1.161.190:38717)
0923 22:07:59.310857 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.310859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.310867 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:07:59.310884 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.310885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.310893 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.310952 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.311311 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.312018 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.312739 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.312913 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.312985 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.313731 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.313732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.313733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.313738 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.314803 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.315100 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.315101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.315101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.315145 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.315382 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.315382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.315383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.315489 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.315566 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.315676 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.315736 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:07:59.315820 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.310668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.129:52945)
0923 22:07:59.310774 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.310783 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.310828 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.310965 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.310965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.311136 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.311219 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.311220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.312111 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.312111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.312626 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.312628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.312985 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.312986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.313041 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.313421 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.313425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.313654 (+ 229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.314886 (+ 1232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.314887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.314888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.315037 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.315268 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.315268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.315269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.315317 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.315564 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.315564 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.315601 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.315657 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.315736 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":59,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:07:59.329959 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38717
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.401222 2994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.401369 2994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.401388 2994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.402356 2994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.402393 2994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.402410 2994 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:59.402450 2994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:07:59.403813 2994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:38717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:59.404011 2994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.404220 2994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.408450 3002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.408422 3000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.408664 2994 server_base.cc:1034] running on GCE node
W20260923 22:07:59.408422 2999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.408931 2994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.409199 2994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.410382 2994 hybrid_clock.cc:648] HybridClock initialized: now 1790201279410367 us; error 28 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.412567 2994 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:07:59.412986 2994 webserver.cc:460] Webserver started at http://127.1.161.130:35351/ using document root <none> and password file <none>
I20260923 22:07:59.413154 2994 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.413189 2994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.413266 2994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.414143 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "31774a1d6203419f87797768a5625806"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "379a31d0d2fb0dbf56a19c787b0e07bd"
server_key_iv: "8bbf33f79239bf0431a1561085d351b2"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.414422 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "31774a1d6203419f87797768a5625806"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "379a31d0d2fb0dbf56a19c787b0e07bd"
server_key_iv: "8bbf33f79239bf0431a1561085d351b2"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.415479 2994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:07:59.416103 3009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.416271 2994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:07:59.416344 2994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "31774a1d6203419f87797768a5625806"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "379a31d0d2fb0dbf56a19c787b0e07bd"
server_key_iv: "8bbf33f79239bf0431a1561085d351b2"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.416394 2994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.443760 2994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.445158 2994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.445245 2994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.445593 2994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.445624 2994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.445643 2994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.445690 2994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.451834 2994 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:38159
I20260923 22:07:59.451907 3124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:38159 every 8 connection(s)
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:59.452138 2994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:07:59.453984 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 2994
I20260923 22:07:59.454071 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:07:59.454253 1670 external_mini_cluster.cc:1428] Setting key 1db01bfaf8d127957c8bb65251242d97
I20260923 22:07:59.454851 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.446017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:47135 (local address 127.1.161.190:38717)
0923 22:07:59.446079 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.446082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.446201 (+ 119us) server_negotiation.cc:407] Connection header received
0923 22:07:59.446567 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.446568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.446579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.446654 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.447339 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.448112 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.449723 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.449997 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.450064 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.452417 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.452418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.452419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.452425 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.453675 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.454007 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.454008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.454008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.454064 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.454265 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.454266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.454266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.454377 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.454434 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.454684 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.454736 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:07:59.455008 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.445946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.130:47135)
0923 22:07:59.446096 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.446127 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.446489 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.446690 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.446693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.447009 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.447257 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.447261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.448210 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.448211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.449626 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.449628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.450078 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.450078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.450225 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.450686 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.450695 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.452332 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.453739 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.453742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.453743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.453933 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.454122 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.454122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.454123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.454225 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.454442 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.454448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.454572 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.454663 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.454788 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260923 22:07:59.455438 3125 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38717
I20260923 22:07:59.455503 3125 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.455605 3125 heartbeater.cc:511] Master 127.1.161.190:38717 requested a full tablet report, sending...
I20260923 22:07:59.455847 2781 ts_manager.cc:194] Registered new tserver with Master: 31774a1d6203419f87797768a5625806 (127.1.161.130:38159)
I20260923 22:07:59.456204 2781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:47135
I20260923 22:07:59.461475 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.456278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.130:45861)
0923 22:07:59.456481 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.456490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.456562 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.456697 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.456697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.456850 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.456898 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.456899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.457605 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.457605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.458006 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.458007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.458414 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.458415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.458484 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.458887 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.458891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.459138 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.460680 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.460681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.460681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.460812 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.460988 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.460989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.460989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.461029 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.461300 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.461300 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.461327 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.461365 (+ 38us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.461421 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260923 22:07:59.461530 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.456404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:45861 (local address 127.1.161.190:38717)
0923 22:07:59.456433 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.456434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.456533 (+ 99us) server_negotiation.cc:407] Connection header received
0923 22:07:59.456619 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.456619 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.456627 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.456651 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.456980 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.457521 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.458077 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.458348 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.458414 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.459255 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.459257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.459257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.459264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.460610 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.460875 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.460875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.460876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.460928 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.461085 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.461086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.461086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.461239 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.461296 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.461388 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.461449 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:07:59.475853 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38717
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:07:59.544538 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:07:59.544713 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:07:59.544746 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:07:59.545713 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:07:59.545770 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:07:59.545800 3134 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:07:59.545833 3134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:07:59.546991 3134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:38717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:07:59.547225 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:07:59.547405 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:07:59.551599 3140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:07:59.551599 3139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.551685 3134 server_base.cc:1034] running on GCE node
W20260923 22:07:59.551687 3142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:07:59.551993 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:07:59.552234 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:07:59.553375 3134 hybrid_clock.cc:648] HybridClock initialized: now 1790201279553373 us; error 33 us; skew 500 ppm
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:07:59.555739 3134 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:07:59.556066 3134 webserver.cc:460] Webserver started at http://127.1.161.131:33975/ using document root <none> and password file <none>
I20260923 22:07:59.556284 3134 fs_manager.cc:359] Metadata directory not provided
I20260923 22:07:59.556344 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:07:59.556453 3134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:07:59.557328 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "38e34b2ba9c94752bc8fc670255f52bc"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "743adeac0bd292c3e48c4751fa60334a"
server_key_iv: "74859704592724b49b171907b182e2c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.557631 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "38e34b2ba9c94752bc8fc670255f52bc"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "743adeac0bd292c3e48c4751fa60334a"
server_key_iv: "74859704592724b49b171907b182e2c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.558744 3134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:07:59.559473 3149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.559671 3134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:59.559782 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "38e34b2ba9c94752bc8fc670255f52bc"
format_stamp: "Formatted at 2026-09-23 22:07:59 on dist-test-slave-npvd"
server_key: "743adeac0bd292c3e48c4751fa60334a"
server_key_iv: "74859704592724b49b171907b182e2c7"
server_key_version: "encryptionkey@0"
I20260923 22:07:59.559849 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:07:59.567471 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:07:59.569247 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:07:59.569391 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:07:59.569845 3134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:07:59.569919 3134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.569957 3134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:07:59.569983 3134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:59.576342 3134 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:33515
I20260923 22:07:59.576610 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:07:59.576647 3264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:33515 every 8 connection(s)
I20260923 22:07:59.578414 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.570279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:45295 (local address 127.1.161.190:38717)
0923 22:07:59.570316 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.570318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.570527 (+ 209us) server_negotiation.cc:407] Connection header received
0923 22:07:59.570948 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.570949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.570973 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.571034 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.571681 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.572220 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.573198 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.573377 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.573448 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.576199 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.576200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.576201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.576207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.577537 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.577822 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.577823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.577823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.577864 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.578030 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.578031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.578031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.578111 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.578152 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.578314 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.578362 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:07:59.578569 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.570276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.131:45295)
0923 22:07:59.570420 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.570447 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.570836 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.571101 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.571104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.571402 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.571590 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.571593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.572322 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.572324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.573111 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.573113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.573455 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.573456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.573565 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.574008 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.574025 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.576094 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.577594 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.577597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.577598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.577764 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.577910 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.577911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.577911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.577982 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.578155 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.578158 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.578239 (+ 81us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.578303 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.578390 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260923 22:07:59.579003 3265 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38717
I20260923 22:07:59.579080 3265 heartbeater.cc:464] Registering TS with master...
I20260923 22:07:59.579101 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 3134
I20260923 22:07:59.579226 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:07:59.579268 3265 heartbeater.cc:511] Master 127.1.161.190:38717 requested a full tablet report, sending...
I20260923 22:07:59.579396 1670 external_mini_cluster.cc:1428] Setting key 5e10f48621f8b8e9cea66d7bd04a1960
I20260923 22:07:59.579612 2782 ts_manager.cc:194] Registered new tserver with Master: 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515)
I20260923 22:07:59.579969 2782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:45295
I20260923 22:07:59.580148 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:07:59.585139 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.580557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.1.161.131:60217)
0923 22:07:59.580662 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.580670 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.580723 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.580818 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.580819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.580966 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:07:59.581002 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.581003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.581518 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.581519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.581957 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.581958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.582284 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.582285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.582351 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.582642 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:07:59.582646 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:07:59.582973 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:07:59.584370 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.584371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.584371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.584524 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.584694 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:07:59.584694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:07:59.584695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:07:59.584736 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:07:59.584951 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:07:59.584951 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:07:59.584987 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.585028 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.585089 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260923 22:07:59.585233 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.580209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60217 (local address 127.1.161.190:38717)
0923 22:07:59.580238 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.580240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.580730 (+ 490us) server_negotiation.cc:407] Connection header received
0923 22:07:59.580767 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.580768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.580777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.580817 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.581066 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.581441 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.582059 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.582216 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.582282 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.583046 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.583047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.583047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.583052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.584295 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.584587 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.584588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.584588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.584635 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.584787 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.584787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.584788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.584901 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.584957 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.585049 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.585117 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:07:59 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:07:59.608788 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.601706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.1.161.190:38717)
0923 22:07:59.601769 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.601772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.601781 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:07:59.601803 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.601804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.601815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.601880 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:07:59.602682 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.603400 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.604042 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.604358 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.604443 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.606790 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:07:59.606791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:07:59.606792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:07:59.606797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:07:59.607906 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.608186 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.608187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.608187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.608228 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:07:59.608355 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:07:59.608356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:07:59.608356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:07:59.608425 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:07:59.608484 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.608681 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.608731 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:07:59.610639 2782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:59.612273 2782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:59.617844 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38159 (local address 127.0.0.1:38034)
0923 22:07:59.614588 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.614624 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.614700 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.614886 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.614889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.614890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.614947 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.614948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.616174 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.616174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.616945 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.616947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.617599 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.617665 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.617676 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.617738 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.617779 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":70,"threads_started":1}
I20260923 22:07:59.617868 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.1.161.130:38159)
0923 22:07:59.614423 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.614427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.614680 (+ 253us) server_negotiation.cc:407] Connection header received
0923 22:07:59.614780 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.614782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.614797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.614867 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.615040 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.616076 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617021 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.617506 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617595 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.617646 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.617752 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.617786 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260923 22:07:59.617964 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46123 (local address 127.0.0.1:35912)
0923 22:07:59.614590 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.614624 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.614700 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.615025 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.615025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.615027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.615063 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.615065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.615783 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.615784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.616869 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.616871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.617729 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.617783 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.617788 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.617832 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.617864 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260923 22:07:59.618196 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.1.161.129:46123)
0923 22:07:59.614588 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.614591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.614679 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:07:59.614945 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.614948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.614963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.615033 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.615120 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.615709 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.616933 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.617576 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617647 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.617694 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.618069 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.618109 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260923 22:07:59.618456 3059 tablet_service.cc:1467] Processing CreateTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 (DEFAULT_TABLE table=test-table [id=2f7e5b3464fe497e9823844651c536c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.618786 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33515 (local address 127.0.0.1:59378)
0923 22:07:59.614803 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.614811 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.614876 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.615120 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.615121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.615123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.615154 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.615155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.616218 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.616219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.617111 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.617113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.618529 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.618531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.618605 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.618610 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.618666 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.618706 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":68,"threads_started":1}
I20260923 22:07:59.618979 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d7926a2ab44a45989ccfdb5f91aa69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.619021 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.614413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59378 (local address 127.1.161.131:33515)
0923 22:07:59.614576 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.614579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.614932 (+ 353us) server_negotiation.cc:407] Connection header received
0923 22:07:59.614977 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.614979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.614994 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.615058 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.615252 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.616136 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617185 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.617777 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.617839 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.617879 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.618890 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.618933 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260923 22:07:59.618955 2919 tablet_service.cc:1467] Processing CreateTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 (DEFAULT_TABLE table=test-table [id=2f7e5b3464fe497e9823844651c536c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.619304 2919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d7926a2ab44a45989ccfdb5f91aa69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.622622 3285 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Bootstrap starting.
I20260923 22:07:59.622923 3199 tablet_service.cc:1467] Processing CreateTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 (DEFAULT_TABLE table=test-table [id=2f7e5b3464fe497e9823844651c536c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.623234 3199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d7926a2ab44a45989ccfdb5f91aa69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.623289 3285 tablet_bootstrap.cc:654] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.623311 3286 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Bootstrap starting.
I20260923 22:07:59.623555 3285 log.cc:824] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.623855 3286 tablet_bootstrap.cc:654] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.624141 3286 log.cc:824] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.624210 3285 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: No bootstrap required, opened a new log
I20260923 22:07:59.624259 3285 ts_tablet_manager.cc:1397] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:07:59.624792 3286 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: No bootstrap required, opened a new log
I20260923 22:07:59.624855 3286 ts_tablet_manager.cc:1397] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:59.625690 3285 raft_consensus.cc:348] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.625839 3285 raft_consensus.cc:374] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.625877 3285 raft_consensus.cc:729] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31774a1d6203419f87797768a5625806, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.625984 3289 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Bootstrap starting.
I20260923 22:07:59.625970 3285 consensus_queue.cc:260] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.626209 3285 ts_tablet_manager.cc:1428] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:07:59.626334 3125 heartbeater.cc:503] Master 127.1.161.190:38717 was elected leader, sending a full tablet report...
I20260923 22:07:59.626555 3286 raft_consensus.cc:348] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.626698 3286 raft_consensus.cc:374] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.626714 3289 tablet_bootstrap.cc:654] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.626734 3286 raft_consensus.cc:729] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7191c4b7f4402c8f7adbfe0fb34292, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.626827 3286 consensus_queue.cc:260] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.626967 3289 log.cc:824] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:07:59.627054 3286 ts_tablet_manager.cc:1428] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.627146 2985 heartbeater.cc:503] Master 127.1.161.190:38717 was elected leader, sending a full tablet report...
I20260923 22:07:59.627619 3289 tablet_bootstrap.cc:492] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: No bootstrap required, opened a new log
I20260923 22:07:59.627672 3289 ts_tablet_manager.cc:1397] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:07:59.628940 3289 raft_consensus.cc:348] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.629050 3289 raft_consensus.cc:374] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.629070 3289 raft_consensus.cc:729] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e34b2ba9c94752bc8fc670255f52bc, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.629190 3289 consensus_queue.cc:260] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.629422 3289 ts_tablet_manager.cc:1428] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:07:59.629467 3265 heartbeater.cc:503] Master 127.1.161.190:38717 was elected leader, sending a full tablet report...
I20260923 22:07:59.657434 3293 raft_consensus.cc:482] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:07:59.657516 3293 raft_consensus.cc:504] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.657754 3293 leader_election.cc:288] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 31774a1d6203419f87797768a5625806 (127.1.161.130:38159)
I20260923 22:07:59.660449 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.657988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:40595 (local address 127.1.161.130:38159)
0923 22:07:59.658019 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.658020 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.658027 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:07:59.658053 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.658053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.658060 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.658120 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.658245 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.658815 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.659829 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.660151 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.660197 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.660202 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.660361 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.660398 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:07:59.660442 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.657889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38159 (local address 127.1.161.131:40595)
0923 22:07:59.657939 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.657950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.657993 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.658123 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.658124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.658125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.658174 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.658178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.658872 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.658873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.659756 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.659758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.660216 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.660217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.660294 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.660296 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.660342 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.660383 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:07:59.660488 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.657969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60117 (local address 127.1.161.129:46123)
0923 22:07:59.658013 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.658015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.658078 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:07:59.658146 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.658147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.658154 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.658199 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:07:59.658336 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.658916 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.659868 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.660162 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.660241 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.660248 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.660398 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.660431 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:07:59.660756 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d7926a2ab44a45989ccfdb5f91aa69" candidate_uuid: "38e34b2ba9c94752bc8fc670255f52bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31774a1d6203419f87797768a5625806" is_pre_election: true
I20260923 22:07:59.660800 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.657890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46123 (local address 127.1.161.131:60117)
0923 22:07:59.658011 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:07:59.658020 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:07:59.658090 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:07:59.658202 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:07:59.658203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:07:59.658214 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:07:59.658276 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.658277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.658989 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.658990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.659769 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:07:59.659770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.660255 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.660258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:07:59.660319 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.660321 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:07:59.660385 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:07:59.660416 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":23,"thread_start_us":55,"threads_started":1}
I20260923 22:07:59.660842 3079 raft_consensus.cc:2454] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38e34b2ba9c94752bc8fc670255f52bc in term 0.
I20260923 22:07:59.661029 3152 leader_election.cc:302] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 38e34b2ba9c94752bc8fc670255f52bc; no voters:
I20260923 22:07:59.661116 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d7926a2ab44a45989ccfdb5f91aa69" candidate_uuid: "38e34b2ba9c94752bc8fc670255f52bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" is_pre_election: true
I20260923 22:07:59.661163 3293 raft_consensus.cc:2781] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:07:59.661206 3293 raft_consensus.cc:482] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:07:59.661223 3293 raft_consensus.cc:3037] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:59.661223 2939 raft_consensus.cc:2454] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38e34b2ba9c94752bc8fc670255f52bc in term 0.
I20260923 22:07:59.661777 3293 raft_consensus.cc:504] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.661885 3293 leader_election.cc:288] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [CANDIDATE]: Term 1 election: Requested vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 31774a1d6203419f87797768a5625806 (127.1.161.130:38159)
I20260923 22:07:59.662014 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d7926a2ab44a45989ccfdb5f91aa69" candidate_uuid: "38e34b2ba9c94752bc8fc670255f52bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31774a1d6203419f87797768a5625806"
I20260923 22:07:59.662074 3079 raft_consensus.cc:3037] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:59.662627 3079 raft_consensus.cc:2454] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38e34b2ba9c94752bc8fc670255f52bc in term 1.
I20260923 22:07:59.662700 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d7926a2ab44a45989ccfdb5f91aa69" candidate_uuid: "38e34b2ba9c94752bc8fc670255f52bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
I20260923 22:07:59.662770 3152 leader_election.cc:302] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 38e34b2ba9c94752bc8fc670255f52bc; no voters:
I20260923 22:07:59.662775 2939 raft_consensus.cc:3037] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:07:59.662839 3293 raft_consensus.cc:2781] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:07:59.662940 3293 raft_consensus.cc:686] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 LEADER]: Becoming Leader. State: Replica: 38e34b2ba9c94752bc8fc670255f52bc, State: Running, Role: LEADER
I20260923 22:07:59.663012 3293 consensus_queue.cc:237] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.663515 2939 raft_consensus.cc:2454] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38e34b2ba9c94752bc8fc670255f52bc in term 1.
I20260923 22:07:59.663686 2781 catalog_manager.cc:5495] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131). New cstate: current_term: 1 leader_uuid: "38e34b2ba9c94752bc8fc670255f52bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } health_report { overall_health: HEALTHY } } }
I20260923 22:07:59.674490 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:07:59.672646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.1.161.131:33515)
0923 22:07:59.672681 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:07:59.672683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:07:59.672710 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:07:59.672766 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:07:59.672766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:07:59.672776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:07:59.672820 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:07:59.672883 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.673222 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.673811 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:07:59.673949 (+ 138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:07:59.673988 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:07:59.674128 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:07:59.674192 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:07:59.674236 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:07:59.674249 (+ 13us) server_negotiation.cc:299] Negotiation successful
0923 22:07:59.674260 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:07:59.675833 3079 raft_consensus.cc:1261] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Refusing update from remote peer 38e34b2ba9c94752bc8fc670255f52bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:59.675853 2939 raft_consensus.cc:1261] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Refusing update from remote peer 38e34b2ba9c94752bc8fc670255f52bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:07:59.676143 3295 consensus_queue.cc:1035] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:59.676229 3293 consensus_queue.cc:1035] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:07:59.678822 3298 mvcc.cc:205] Tried to move back new op lower bound from 7332664441550233600 to 7332664441500381184. Current Snapshot: MvccSnapshot[applied={T|T < 7332664441550233600 or (T in {7332664441550233600})}]
I20260923 22:07:59.679150 3299 mvcc.cc:205] Tried to move back new op lower bound from 7332664441550233600 to 7332664441500381184. Current Snapshot: MvccSnapshot[applied={T|T < 7332664441550233600 or (T in {7332664441550233600})}]
I20260923 22:07:59.680797 2781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:59.680888 2781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:07:59.681460 2781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Sending DeleteTablet for 3 replicas of tablet a0d7926a2ab44a45989ccfdb5f91aa69
I20260923 22:07:59.681748 2919 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:59 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260923 22:07:59.681754 3059 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:59 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38034
I20260923 22:07:59.681903 3199 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d7926a2ab44a45989ccfdb5f91aa69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:07:59 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:59378
I20260923 22:07:59.681931 2781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:07:59.682032 2781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:07:59.682078 3310 tablet_replica.cc:330] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: stopping tablet replica
I20260923 22:07:59.682155 3310 raft_consensus.cc:2229] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:07:59.682222 3310 raft_consensus.cc:2258] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:59.682477 3308 tablet_replica.cc:330] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: stopping tablet replica
I20260923 22:07:59.682545 3310 ts_tablet_manager.cc:1905] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:59.682564 3308 raft_consensus.cc:2229] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:59.682644 3308 raft_consensus.cc:2258] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:59.683374 3199 tablet_service.cc:1467] Processing CreateTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 (DEFAULT_TABLE table=test-table [id=efd3944cc64e4321bb85fb41381dd9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.683380 3059 tablet_service.cc:1467] Processing CreateTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 (DEFAULT_TABLE table=test-table [id=efd3944cc64e4321bb85fb41381dd9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.683454 3199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1daebc7865b4dbaaaa16251ce3904a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.683467 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1daebc7865b4dbaaaa16251ce3904a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.683894 3310 ts_tablet_manager.cc:1918] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:59.683947 3310 log.cc:1196] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/a0d7926a2ab44a45989ccfdb5f91aa69
I20260923 22:07:59.684170 3310 ts_tablet_manager.cc:1939] T a0d7926a2ab44a45989ccfdb5f91aa69 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting consensus metadata
I20260923 22:07:59.684490 3289 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Bootstrap starting.
I20260923 22:07:59.684553 3285 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Bootstrap starting.
I20260923 22:07:59.684679 2768 catalog_manager.cc:4841] TS 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515): tablet a0d7926a2ab44a45989ccfdb5f91aa69 (table test-table [id=2f7e5b3464fe497e9823844651c536c3]) successfully deleted
I20260923 22:07:59.684815 3289 tablet_bootstrap.cc:654] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.685055 3308 ts_tablet_manager.cc:1905] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:59.685180 3285 tablet_bootstrap.cc:654] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.685674 3285 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: No bootstrap required, opened a new log
I20260923 22:07:59.685726 3285 ts_tablet_manager.cc:1397] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:07:59.685858 3285 raft_consensus.cc:348] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.685916 3285 raft_consensus.cc:374] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.685940 3285 raft_consensus.cc:729] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31774a1d6203419f87797768a5625806, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.685972 3285 consensus_queue.cc:260] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.686035 3285 ts_tablet_manager.cc:1428] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:07:59.686329 3308 ts_tablet_manager.cc:1918] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:59.686388 3308 log.cc:1196] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/a0d7926a2ab44a45989ccfdb5f91aa69
I20260923 22:07:59.686621 3308 ts_tablet_manager.cc:1939] T a0d7926a2ab44a45989ccfdb5f91aa69 P 31774a1d6203419f87797768a5625806: Deleting consensus metadata
I20260923 22:07:59.686962 2768 catalog_manager.cc:4841] TS 31774a1d6203419f87797768a5625806 (127.1.161.130:38159): tablet a0d7926a2ab44a45989ccfdb5f91aa69 (table test-table [id=2f7e5b3464fe497e9823844651c536c3]) successfully deleted
I20260923 22:07:59.686964 2919 tablet_service.cc:1467] Processing CreateTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 (DEFAULT_TABLE table=test-table [id=efd3944cc64e4321bb85fb41381dd9e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:07:59.687068 2919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1daebc7865b4dbaaaa16251ce3904a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:07:59.687129 3309 tablet_replica.cc:330] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: stopping tablet replica
I20260923 22:07:59.687219 3309 raft_consensus.cc:2229] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:07:59.687310 3309 raft_consensus.cc:2258] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:07:59.687677 3309 ts_tablet_manager.cc:1905] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:07:59.688064 3286 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Bootstrap starting.
I20260923 22:07:59.688354 3286 tablet_bootstrap.cc:654] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Neither blocks nor log segments found. Creating new log.
I20260923 22:07:59.688979 3286 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: No bootstrap required, opened a new log
I20260923 22:07:59.688994 3309 ts_tablet_manager.cc:1918] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:07:59.689038 3286 ts_tablet_manager.cc:1397] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:07:59.689054 3309 log.cc:1196] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/a0d7926a2ab44a45989ccfdb5f91aa69
I20260923 22:07:59.689174 3286 raft_consensus.cc:348] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.689248 3286 raft_consensus.cc:374] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.689275 3286 raft_consensus.cc:729] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7191c4b7f4402c8f7adbfe0fb34292, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.689284 3309 ts_tablet_manager.cc:1939] T a0d7926a2ab44a45989ccfdb5f91aa69 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting consensus metadata
I20260923 22:07:59.689322 3286 consensus_queue.cc:260] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.689407 3286 ts_tablet_manager.cc:1428] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:07:59.689635 2769 catalog_manager.cc:4841] TS 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123): tablet a0d7926a2ab44a45989ccfdb5f91aa69 (table test-table [id=2f7e5b3464fe497e9823844651c536c3]) successfully deleted
I20260923 22:07:59.691602 3289 tablet_bootstrap.cc:492] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: No bootstrap required, opened a new log
I20260923 22:07:59.691664 3289 ts_tablet_manager.cc:1397] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260923 22:07:59.692179 3289 raft_consensus.cc:348] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.692225 3289 raft_consensus.cc:374] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:07:59.692238 3289 raft_consensus.cc:729] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e34b2ba9c94752bc8fc670255f52bc, State: Initialized, Role: FOLLOWER
I20260923 22:07:59.692297 3289 consensus_queue.cc:260] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:07:59.692395 3289 ts_tablet_manager.cc:1428] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260923 22:07:59.702724 3126 tablet.cc:2369] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:07:59.807384 2986 tablet.cc:2369] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:07:59.827306 3266 tablet.cc:2369] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:00.013005 3290 raft_consensus.cc:482] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:00.013103 3290 raft_consensus.cc:504] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.013433 3290 leader_election.cc:288] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515)
I20260923 22:08:00.015919 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.013661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:48193 (local address 127.1.161.131:33515)
0923 22:08:00.013715 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.013717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.013828 (+ 111us) server_negotiation.cc:407] Connection header received
0923 22:08:00.013876 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.013877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.013893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.013947 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:00.014080 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.014480 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015306 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.015630 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015678 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.015684 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.015819 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.015853 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:08:00.015924 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.013608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33515 (local address 127.1.161.130:48193)
0923 22:08:00.013736 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.013746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.013826 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.013952 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.013953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.013955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:00.014018 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.014020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.014545 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.014546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.015239 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.015240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.015699 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.015760 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.015763 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.015806 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.015831 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":12,"thread_start_us":43,"threads_started":1}
I20260923 22:08:00.015993 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.013673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:49463 (local address 127.1.161.129:46123)
0923 22:08:00.013712 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.013714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.013805 (+ 91us) server_negotiation.cc:407] Connection header received
0923 22:08:00.013874 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.013875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.013883 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.013944 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:00.014110 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.014496 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015360 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.015675 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015735 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.015741 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.015892 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.015928 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:00.016039 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.013606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46123 (local address 127.1.161.130:49463)
0923 22:08:00.013738 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.013746 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.013819 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.013959 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.013960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.013962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:00.014027 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.014028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.014606 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.014607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.015286 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.015289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.015747 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.015748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.015827 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.015830 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.015877 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.015917 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
I20260923 22:08:00.016207 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1daebc7865b4dbaaaa16251ce3904a9" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e34b2ba9c94752bc8fc670255f52bc" is_pre_election: true
I20260923 22:08:00.016243 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1daebc7865b4dbaaaa16251ce3904a9" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" is_pre_election: true
I20260923 22:08:00.016316 3219 raft_consensus.cc:2454] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 0.
I20260923 22:08:00.016320 2939 raft_consensus.cc:2454] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 0.
I20260923 22:08:00.016456 3013 leader_election.cc:302] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 6e7191c4b7f4402c8f7adbfe0fb34292; no voters:
I20260923 22:08:00.016571 3290 raft_consensus.cc:2781] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:00.016604 3290 raft_consensus.cc:482] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:00.016624 3290 raft_consensus.cc:3037] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.017222 3290 raft_consensus.cc:504] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.017336 3290 leader_election.cc:288] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515)
I20260923 22:08:00.017455 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1daebc7865b4dbaaaa16251ce3904a9" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
I20260923 22:08:00.017486 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1daebc7865b4dbaaaa16251ce3904a9" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e34b2ba9c94752bc8fc670255f52bc"
I20260923 22:08:00.017542 2939 raft_consensus.cc:3037] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.017589 3219 raft_consensus.cc:3037] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.018167 2939 raft_consensus.cc:2454] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 1.
I20260923 22:08:00.018231 3219 raft_consensus.cc:2454] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 1.
I20260923 22:08:00.018318 3013 leader_election.cc:302] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 6e7191c4b7f4402c8f7adbfe0fb34292; no voters:
I20260923 22:08:00.018417 3290 raft_consensus.cc:2781] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:00.018533 3290 raft_consensus.cc:686] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 1 LEADER]: Becoming Leader. State: Replica: 31774a1d6203419f87797768a5625806, State: Running, Role: LEADER
I20260923 22:08:00.018615 3290 consensus_queue.cc:237] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.019263 2781 catalog_manager.cc:5495] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31774a1d6203419f87797768a5625806 (127.1.161.130). New cstate: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:08:00 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:00.057368 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.051691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.1.161.190:38717)
0923 22:08:00.051742 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.051744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.051834 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:08:00.051897 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.051897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.051906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.051963 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.052332 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.052713 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.053278 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.053429 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.053497 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.055212 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.055213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.055213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:00.055219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.056386 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.056699 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.056701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.056701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.056745 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.056915 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.056915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.056916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.057008 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.057073 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.057202 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.057276 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:08:00.057432 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.051649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38717 (local address 127.0.0.1:46420)
0923 22:08:00.051769 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.051783 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.051849 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.051966 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.051967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.052223 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:00.052260 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.052262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.052788 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.052789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.053209 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.053211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.053498 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.053499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.053561 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.053970 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:00.053973 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:00.055100 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:00.056464 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:00.056470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:00.056472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:00.056628 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:00.056803 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:00.056804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:00.056804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:00.056860 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:00.057072 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:00.057079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:00.057119 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.057183 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.057321 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":60,"thread_start_us":65,"threads_started":1}
I20260923 22:08:00.059068 2781 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:46420:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:08:00.059311 2781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:00.060518 3059 tablet_service.cc:1467] Processing CreateTablet for tablet 7a80a4354a0a41e09df1e7313d471e3a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be417c070352433da153bd64a72673cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:00.060509 2919 tablet_service.cc:1467] Processing CreateTablet for tablet 7a80a4354a0a41e09df1e7313d471e3a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be417c070352433da153bd64a72673cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:00.060536 3199 tablet_service.cc:1467] Processing CreateTablet for tablet 7a80a4354a0a41e09df1e7313d471e3a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be417c070352433da153bd64a72673cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:00.060653 2919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a80a4354a0a41e09df1e7313d471e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.060654 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a80a4354a0a41e09df1e7313d471e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.060658 3199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a80a4354a0a41e09df1e7313d471e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:00.061631 3285 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: Bootstrap starting.
I20260923 22:08:00.061903 3289 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: Bootstrap starting.
I20260923 22:08:00.061902 3286 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: Bootstrap starting.
I20260923 22:08:00.061985 3285 tablet_bootstrap.cc:654] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.062220 3289 tablet_bootstrap.cc:654] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.062230 3286 tablet_bootstrap.cc:654] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:00.062572 3285 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: No bootstrap required, opened a new log
I20260923 22:08:00.062629 3285 ts_tablet_manager.cc:1397] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:00.062742 3286 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: No bootstrap required, opened a new log
I20260923 22:08:00.062778 3286 ts_tablet_manager.cc:1397] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:00.062757 3285 raft_consensus.cc:348] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.062824 3285 raft_consensus.cc:374] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.062842 3285 raft_consensus.cc:729] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31774a1d6203419f87797768a5625806, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.062875 3285 consensus_queue.cc:260] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.062903 3286 raft_consensus.cc:348] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.062947 3285 ts_tablet_manager.cc:1428] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.062958 3286 raft_consensus.cc:374] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.062973 3286 raft_consensus.cc:729] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7191c4b7f4402c8f7adbfe0fb34292, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.062974 3290 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.063006 3286 consensus_queue.cc:260] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.063033 3290 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.063073 3286 ts_tablet_manager.cc:1428] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.063278 3289 tablet_bootstrap.cc:492] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: No bootstrap required, opened a new log
I20260923 22:08:00.063265 3291 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.063334 3289 ts_tablet_manager.cc:1397] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:00.063340 3291 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.063438 3289 raft_consensus.cc:348] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.063503 3289 raft_consensus.cc:374] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:00.063522 3289 raft_consensus.cc:729] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e34b2ba9c94752bc8fc670255f52bc, State: Initialized, Role: FOLLOWER
I20260923 22:08:00.063555 3289 consensus_queue.cc:260] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.063642 3289 ts_tablet_manager.cc:1428] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:00.063668 3303 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.063752 3303 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.072681 3290 raft_consensus.cc:482] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:00.072757 3290 raft_consensus.cc:504] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.072870 3290 leader_election.cc:288] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515)
I20260923 22:08:00.072945 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a80a4354a0a41e09df1e7313d471e3a" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" is_pre_election: true
I20260923 22:08:00.072999 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a80a4354a0a41e09df1e7313d471e3a" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e34b2ba9c94752bc8fc670255f52bc" is_pre_election: true
I20260923 22:08:00.073024 2939 raft_consensus.cc:2454] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 0.
I20260923 22:08:00.073061 3219 raft_consensus.cc:2454] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 0.
I20260923 22:08:00.073132 3013 leader_election.cc:302] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 6e7191c4b7f4402c8f7adbfe0fb34292; no voters:
I20260923 22:08:00.073194 3290 raft_consensus.cc:2781] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:00.073225 3290 raft_consensus.cc:482] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:00.073251 3290 raft_consensus.cc:3037] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.073685 3290 raft_consensus.cc:504] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.073797 3290 leader_election.cc:288] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123), 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515)
I20260923 22:08:00.073894 2939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a80a4354a0a41e09df1e7313d471e3a" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292"
I20260923 22:08:00.073922 3219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a80a4354a0a41e09df1e7313d471e3a" candidate_uuid: "31774a1d6203419f87797768a5625806" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38e34b2ba9c94752bc8fc670255f52bc"
I20260923 22:08:00.073946 2939 raft_consensus.cc:3037] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.073968 3219 raft_consensus.cc:3037] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:00.074497 2939 raft_consensus.cc:2454] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 1.
I20260923 22:08:00.074497 3219 raft_consensus.cc:2454] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31774a1d6203419f87797768a5625806 in term 1.
I20260923 22:08:00.074683 3013 leader_election.cc:302] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31774a1d6203419f87797768a5625806, 6e7191c4b7f4402c8f7adbfe0fb34292; no voters:
I20260923 22:08:00.074788 3290 raft_consensus.cc:2781] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:00.074843 3290 raft_consensus.cc:686] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [term 1 LEADER]: Becoming Leader. State: Replica: 31774a1d6203419f87797768a5625806, State: Running, Role: LEADER
I20260923 22:08:00.074867 3290 consensus_queue.cc:237] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } }
I20260923 22:08:00.075094 3316 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: TxnStatusTablet state changed. Reason: New leader 31774a1d6203419f87797768a5625806. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.075220 3316 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:00.075318 2781 catalog_manager.cc:5495] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31774a1d6203419f87797768a5625806 (127.1.161.130). New cstate: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:00.075366 3328 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:08:00.075440 3328 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:08:00.080771 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.078383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.1.161.130:38159)
0923 22:08:00.078419 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.078420 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.078428 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:00.078480 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.078480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.078493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.078536 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:00.078619 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.079210 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.080008 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.080237 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.080305 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.080467 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.080555 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.080637 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.080685 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.080701 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:00.080773 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.078313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38159 (local address 127.0.0.1:38038)
0923 22:08:00.078370 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.078378 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.078413 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.078547 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.078549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.078550 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:00.078578 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.078580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.079285 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.079286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.079929 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.079931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.080307 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.080308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.080361 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.080374 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.080627 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.080635 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.080680 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.080714 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:08:00.081612 2939 raft_consensus.cc:1261] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Refusing update from remote peer 31774a1d6203419f87797768a5625806: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.081683 3219 raft_consensus.cc:1261] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Refusing update from remote peer 31774a1d6203419f87797768a5625806: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.081905 3290 consensus_queue.cc:1035] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.082015 3316 consensus_queue.cc:1035] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.082302 3291 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: TxnStatusTablet state changed. Reason: New leader 31774a1d6203419f87797768a5625806. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.082368 3291 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.082705 3303 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: TxnStatusTablet state changed. Reason: New leader 31774a1d6203419f87797768a5625806. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.082784 3303 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.083122 3290 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.083217 3290 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:00.083426 3300 mvcc.cc:205] Tried to move back new op lower bound from 7332664443212664832 to 7332664443186782208. Current Snapshot: MvccSnapshot[applied={T|T < 7332664443212664832 or (T in {7332664443212664832})}]
I20260923 22:08:00.083802 3303 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.083868 3303 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 38e34b2ba9c94752bc8fc670255f52bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.084583 3316 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.084666 3316 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 31774a1d6203419f87797768a5625806: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:00.084781 3291 tablet_replica.cc:439] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31774a1d6203419f87797768a5625806" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 } } peers { permanent_uuid: "31774a1d6203419f87797768a5625806" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38159 } } peers { permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 } } }
I20260923 22:08:00.084839 3291 tablet_replica.cc:442] T 7a80a4354a0a41e09df1e7313d471e3a P 6e7191c4b7f4402c8f7adbfe0fb34292: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:00.086558 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.084554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.1.161.130:38159)
0923 22:08:00.084594 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.084596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.084603 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:00.084664 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.084665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.084674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.084713 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:00.084856 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.085429 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.086028 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.086213 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.086275 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.086380 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.086405 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.086442 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.086489 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.086502 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:08:00.094924 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.088600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:43533 (local address 127.1.161.130:38159)
0923 22:08:00.088631 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.088632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.092852 (+ 4220us) server_negotiation.cc:407] Connection header received
0923 22:08:00.092891 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.092892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.092898 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.092936 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:00.093082 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.093443 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.094374 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.094531 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.094581 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.094737 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.094755 (+ 18us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.094786 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.094852 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.094864 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:00.094991 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.088535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38159 (local address 127.1.161.130:43533)
0923 22:08:00.092776 (+ 4241us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:00.092785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:00.092845 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:00.092953 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:00.092954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:00.092963 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:00.093027 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.093028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.093510 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.093510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.094307 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:00.094309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.094597 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.094598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:00.094661 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.094672 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:00.094802 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:00.094803 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:00.094843 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:08:00.094874 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4211,"thread_start_us":92,"threads_started":1}
I20260923 22:08:00.096653 3219 raft_consensus.cc:1261] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Refusing update from remote peer 31774a1d6203419f87797768a5625806: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.096680 2939 raft_consensus.cc:1261] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Refusing update from remote peer 31774a1d6203419f87797768a5625806: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:00.096809 3336 consensus_queue.cc:1035] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38e34b2ba9c94752bc8fc670255f52bc" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.096868 3316 consensus_queue.cc:1035] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7191c4b7f4402c8f7adbfe0fb34292" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:00.106853 2782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:00.106935 2782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:00.107482 2782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72a09490abc946ba867acfadbafbf1ea: Sending DeleteTablet for 3 replicas of tablet d1daebc7865b4dbaaaa16251ce3904a9
I20260923 22:08:00.107707 2919 tablet_service.cc:1514] Processing DeleteTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35912
I20260923 22:08:00.107707 3059 tablet_service.cc:1514] Processing DeleteTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38034
I20260923 22:08:00.107712 3199 tablet_service.cc:1514] Processing DeleteTablet for tablet d1daebc7865b4dbaaaa16251ce3904a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:00 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:59378
I20260923 22:08:00.107787 3309 tablet_replica.cc:330] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: stopping tablet replica
I20260923 22:08:00.107789 3310 tablet_replica.cc:330] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: stopping tablet replica
I20260923 22:08:00.107829 3309 raft_consensus.cc:2229] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:00.107831 3310 raft_consensus.cc:2229] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:00.107859 3310 raft_consensus.cc:2258] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.107859 3309 raft_consensus.cc:2258] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.108203 3308 tablet_replica.cc:330] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: stopping tablet replica
I20260923 22:08:00.108211 3309 ts_tablet_manager.cc:1905] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:00.108258 3308 raft_consensus.cc:2229] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:00.108369 3308 raft_consensus.cc:2258] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:00.109107 3308 ts_tablet_manager.cc:1905] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:00.109644 3309 ts_tablet_manager.cc:1918] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:00.109695 3309 log.cc:1196] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/d1daebc7865b4dbaaaa16251ce3904a9
I20260923 22:08:00.109905 3309 ts_tablet_manager.cc:1939] T d1daebc7865b4dbaaaa16251ce3904a9 P 6e7191c4b7f4402c8f7adbfe0fb34292: Deleting consensus metadata
I20260923 22:08:00.110416 3310 ts_tablet_manager.cc:1905] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:00.110636 3308 ts_tablet_manager.cc:1918] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:00.110702 3308 log.cc:1196] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/d1daebc7865b4dbaaaa16251ce3904a9
I20260923 22:08:00.110909 3308 ts_tablet_manager.cc:1939] T d1daebc7865b4dbaaaa16251ce3904a9 P 31774a1d6203419f87797768a5625806: Deleting consensus metadata
I20260923 22:08:00.111219 2769 catalog_manager.cc:4841] TS 6e7191c4b7f4402c8f7adbfe0fb34292 (127.1.161.129:46123): tablet d1daebc7865b4dbaaaa16251ce3904a9 (table test-table [id=efd3944cc64e4321bb85fb41381dd9e9]) successfully deleted
I20260923 22:08:00.111862 3310 ts_tablet_manager.cc:1918] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:00.111919 3310 log.cc:1196] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/d1daebc7865b4dbaaaa16251ce3904a9
I20260923 22:08:00.112110 3310 ts_tablet_manager.cc:1939] T d1daebc7865b4dbaaaa16251ce3904a9 P 38e34b2ba9c94752bc8fc670255f52bc: Deleting consensus metadata
I20260923 22:08:00.112279 2768 catalog_manager.cc:4841] TS 31774a1d6203419f87797768a5625806 (127.1.161.130:38159): tablet d1daebc7865b4dbaaaa16251ce3904a9 (table test-table [id=efd3944cc64e4321bb85fb41381dd9e9]) successfully deleted
I20260923 22:08:00.112465 2768 catalog_manager.cc:4841] TS 38e34b2ba9c94752bc8fc670255f52bc (127.1.161.131:33515): tablet d1daebc7865b4dbaaaa16251ce3904a9 (table test-table [id=efd3944cc64e4321bb85fb41381dd9e9]) successfully deleted
Sep 23 22:08:00 dist-test-slave-npvd krb5kdc[2737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:08:00.122937 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.108344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.1.161.129:46123)
0923 22:08:00.108377 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.108379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.115352 (+ 6973us) server_negotiation.cc:407] Connection header received
0923 22:08:00.115393 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.115394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.115403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.115466 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.115944 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.116640 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.117284 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.117569 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.117631 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.120566 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.120573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.120573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:00.120578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.121986 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.122331 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.122333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.122334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.122379 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.122550 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.122551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.122551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.122658 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.122746 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.122836 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.122886 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:08:00.123332 2979 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35916
I20260923 22:08:00.129320 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:00.124134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.1.161.190:38717)
0923 22:08:00.124175 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:00.124177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:00.124209 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:08:00.124274 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:00.124275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:00.124283 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:00.124332 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:00.124763 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.125186 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.125779 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:00.125969 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:00.126043 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:00.127043 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:00.127044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:00.127044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:00.127051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:00.128238 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.128566 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.128567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.128567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.128628 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:00.128841 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:00.128842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:00.128843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:00.128951 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:00.129050 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:00.129157 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 22:08:00.129226 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260923 22:08:00.129523 2782 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46422
I20260923 22:08:00.130051 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2854
I20260923 22:08:00.136006 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2994
I20260923 22:08:00.140606 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 3134
I20260923 22:08:00.144899 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 2750
2026-09-23T22:08:00Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2121 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:00 dist-test-slave-npvd krb5kdc[3347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:00 dist-test-slave-npvd krb5kdc[3347](info): set up 2 sockets
Sep 23 22:08:00 dist-test-slave-npvd krb5kdc[3347](info): commencing operation
krb5kdc: starting...
W20260923 22:08:01.058619 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.893s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:01.154917 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40349
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:01.222748 3360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.222896 3360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.222915 3360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.223886 3360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:01.223919 3360 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:08:01.223930 3360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.223941 3360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.223989 3360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:01.224002 3360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:01.225203 3360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40349
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:01.225399 3360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.225579 3360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.229667 3365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.229682 3368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.229669 3366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.229898 3360 server_base.cc:1034] running on GCE node
I20260923 22:08:01.230175 3360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.230440 3360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.231587 3360 hybrid_clock.cc:648] HybridClock initialized: now 1790201281231573 us; error 25 us; skew 500 ppm
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:01.233603 3360 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:01.233978 3360 webserver.cc:460] Webserver started at http://127.1.161.190:35745/ using document root <none> and password file <none>
I20260923 22:08:01.234202 3360 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.234267 3360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.234409 3360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.235363 3360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "3141988640154b99af3c9f0f30e020d0"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "5a36c47effd29def3a008c734fb8bd6b"
server_key_iv: "4fda21a2eb5df605140fe40211aa30ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.235651 3360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "3141988640154b99af3c9f0f30e020d0"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "5a36c47effd29def3a008c734fb8bd6b"
server_key_iv: "4fda21a2eb5df605140fe40211aa30ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.236683 3360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.237262 3375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.237394 3360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:01.237478 3360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "3141988640154b99af3c9f0f30e020d0"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "5a36c47effd29def3a008c734fb8bd6b"
server_key_iv: "4fda21a2eb5df605140fe40211aa30ec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.237545 3360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.256080 3360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.257441 3360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.257578 3360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.261611 3360 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40349
I20260923 22:08:01.261672 3437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40349 every 8 connection(s)
I20260923 22:08:01.261922 3360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:01.262480 3438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.264119 3438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Bootstrap starting.
I20260923 22:08:01.264539 3438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.264727 3438 log.cc:824] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.265241 3438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: No bootstrap required, opened a new log
I20260923 22:08:01.266992 3438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } }
I20260923 22:08:01.267109 3438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.267130 3438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3141988640154b99af3c9f0f30e020d0, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.267266 3438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } }
I20260923 22:08:01.267342 3438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:01.267364 3438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:01.267390 3438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.267824 3438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } }
I20260923 22:08:01.267899 3438 leader_election.cc:302] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3141988640154b99af3c9f0f30e020d0; no voters:
I20260923 22:08:01.268005 3438 leader_election.cc:288] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:01.268070 3441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.268203 3438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:01.268369 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 3360
I20260923 22:08:01.268445 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:01.268626 1670 external_mini_cluster.cc:1428] Setting key 701cee54d5f8b7c5102aa65965929741
I20260923 22:08:01.268229 3441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3141988640154b99af3c9f0f30e020d0, State: Running, Role: LEADER
I20260923 22:08:01.268848 3441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } }
I20260923 22:08:01.269557 3443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3141988640154b99af3c9f0f30e020d0. Latest consensus state: current_term: 1 leader_uuid: "3141988640154b99af3c9f0f30e020d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } } }
I20260923 22:08:01.269761 3442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3141988640154b99af3c9f0f30e020d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3141988640154b99af3c9f0f30e020d0" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40349 } } }
I20260923 22:08:01.269806 3443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:01.269814 3442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0 [sys.catalog]: This master's current role is: LEADER
W20260923 22:08:01.269791 3457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:01.270026 3457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:01.270120 3459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:01.270457 3459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:01.271809 3459 catalog_manager.cc:1347] Generated new cluster ID: fd09d0afa74a4fcfa262e73e9175da36
I20260923 22:08:01.271859 3459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.277637 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.268970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.1.161.190:40349)
0923 22:08:01.269195 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.269205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.269245 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:08:01.269424 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.269434 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.269495 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.269603 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.270761 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.271924 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.272565 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.272906 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.272999 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.274869 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.274872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.274872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.274881 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.276274 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.276615 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.276618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.276618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.276722 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.276903 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.276904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.276904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.277127 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.277277 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.277345 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.277464 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260923 22:08:01.279774 3459 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:01.280227 3459 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:01.286602 3459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Generated new TSK 0
I20260923 22:08:01.286726 3459 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:01.312024 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40349
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:01.385285 3464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.385464 3464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.385488 3464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.386412 3464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.386446 3464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.386466 3464 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:08:01.386476 3464 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:01.386492 3464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:01.387727 3464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:01.387889 3464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.388036 3464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.392516 3470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.392566 3469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.392659 3464 server_base.cc:1034] running on GCE node
W20260923 22:08:01.392516 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.392891 3464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.393064 3464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.394222 3464 hybrid_clock.cc:648] HybridClock initialized: now 1790201281394218 us; error 31 us; skew 500 ppm
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:01.396466 3464 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:01.396763 3464 webserver.cc:460] Webserver started at http://127.1.161.129:37065/ using document root <none> and password file <none>
I20260923 22:08:01.396914 3464 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.396946 3464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.397017 3464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.397779 3464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "fcb10a317ad34e5fb5c436b5b250e190"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "af2b2b0912ca794ac61fa1a0b1197687"
server_key_iv: "49b26b35f1ba13a6bd2267fd9e5b2fec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.398030 3464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "fcb10a317ad34e5fb5c436b5b250e190"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "af2b2b0912ca794ac61fa1a0b1197687"
server_key_iv: "49b26b35f1ba13a6bd2267fd9e5b2fec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.399026 3464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:01.399760 3479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.399914 3464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.399978 3464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "fcb10a317ad34e5fb5c436b5b250e190"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "af2b2b0912ca794ac61fa1a0b1197687"
server_key_iv: "49b26b35f1ba13a6bd2267fd9e5b2fec"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.400024 3464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.409823 3464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.411844 3464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.411952 3464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.412271 3464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:01.412298 3464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.412340 3464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:01.412374 3464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.418541 3464 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:42749
I20260923 22:08:01.418599 3594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:42749 every 8 connection(s)
I20260923 22:08:01.418855 3464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:01.420944 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.412675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:42701 (local address 127.1.161.190:40349)
0923 22:08:01.412733 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.412736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.412926 (+ 190us) server_negotiation.cc:407] Connection header received
0923 22:08:01.413277 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.413278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.413294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.413366 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.414053 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.414778 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.415834 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.416077 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.416157 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.418500 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.418502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.418503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.418509 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.419928 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.420272 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.420272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.420273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.420318 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.420510 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.420511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.420511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.420595 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.420639 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.420838 (+ 199us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.420891 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:01.421048 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.412666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.129:42701)
0923 22:08:01.412784 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.412814 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.413197 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.413406 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.413409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.413701 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.413960 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.413965 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.414867 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.414868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.415755 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.415757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.416174 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.416175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.416284 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.416656 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.416663 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.418392 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.420009 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.420013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.420014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.420211 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.420356 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.420356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.420357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.420466 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.420645 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.420649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.420737 (+ 88us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.420815 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.420903 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260923 22:08:01.421463 3595 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40349
I20260923 22:08:01.421543 3595 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:01.421700 3595 heartbeater.cc:511] Master 127.1.161.190:40349 requested a full tablet report, sending...
I20260923 22:08:01.422191 3391 ts_manager.cc:194] Registered new tserver with Master: fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749)
I20260923 22:08:01.422991 3391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:42701
I20260923 22:08:01.425570 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 3464
I20260923 22:08:01.425680 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:01.425843 1670 external_mini_cluster.cc:1428] Setting key 8501012338e05360ec358b8a9b335cad
I20260923 22:08:01.426965 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.422462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:51663 (local address 127.1.161.190:40349)
0923 22:08:01.422494 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.422496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.422504 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.422522 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.422522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.422529 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.422570 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.422882 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.423459 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.424117 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.424259 (+ 142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.424299 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.425011 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.425012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.425012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.425016 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.426118 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.426360 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.426360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.426361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.426408 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.426596 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.426597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.426597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.426690 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.426737 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.426869 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.426921 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:01.427081 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.422310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.129:51663)
0923 22:08:01.422430 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.422440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.422486 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.422584 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.422585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.422773 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.422828 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.422829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.423549 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.423550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.424058 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.424059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.424315 (+ 256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.424316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.424361 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.424717 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.424721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.424953 (+ 232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.426175 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.426175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.426176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.426303 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.426473 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.426474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.426475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.426531 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.426751 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.426751 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.426791 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.426854 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.426948 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":56,"threads_started":1}
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:01.447363 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40349
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:01.516225 3604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.516374 3604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.516394 3604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.517278 3604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.517305 3604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.517323 3604 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:08:01.517333 3604 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:01.517346 3604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:01.518455 3604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:01.518586 3604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.518714 3604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:01.523031 3609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.523136 3612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.523065 3604 server_base.cc:1034] running on GCE node
W20260923 22:08:01.523030 3610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.523363 3604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.523522 3604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.524642 3604 hybrid_clock.cc:648] HybridClock initialized: now 1790201281524630 us; error 27 us; skew 500 ppm
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:01.526674 3604 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:01.527029 3604 webserver.cc:460] Webserver started at http://127.1.161.130:36515/ using document root <none> and password file <none>
I20260923 22:08:01.527254 3604 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.527311 3604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.527421 3604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.528214 3604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "cbd5cffbaec4467c817942dcdf60a248"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "e017ca03866e44006e7ba4cc78ccfee2"
server_key_iv: "678499ff819709d7c99ade7ce58835e3"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.528522 3604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "cbd5cffbaec4467c817942dcdf60a248"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "e017ca03866e44006e7ba4cc78ccfee2"
server_key_iv: "678499ff819709d7c99ade7ce58835e3"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.529879 3604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:01.530598 3619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.530784 3604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:01.530867 3604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "cbd5cffbaec4467c817942dcdf60a248"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "e017ca03866e44006e7ba4cc78ccfee2"
server_key_iv: "678499ff819709d7c99ade7ce58835e3"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.530943 3604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.545608 3604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.546990 3604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.547123 3604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.547591 3604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:01.547644 3604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.547675 3604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:01.547698 3604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.554118 3604 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:35881
I20260923 22:08:01.554196 3734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:35881 every 8 connection(s)
I20260923 22:08:01.554430 3604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:01.555819 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.547918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:45697 (local address 127.1.161.190:40349)
0923 22:08:01.547965 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.547968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.548173 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:08:01.548527 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.548528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.548540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.548611 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.549030 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.549850 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.551390 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.551609 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.551694 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.553532 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.553534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.553534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.553542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.554956 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.555279 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.555280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.555280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.555324 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.555459 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.555459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.555459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.555536 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.555575 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.555717 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.555765 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:01.555974 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.547912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.130:45697)
0923 22:08:01.548068 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.548098 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.548441 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.548647 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.548650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.548830 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.548971 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.548973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.549965 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.549966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.551310 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.551312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.551715 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.551715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.551832 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.552292 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.552299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.553447 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.555010 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.555012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.555013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.555221 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.555358 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.555358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.555358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.555427 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.555566 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.555568 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.555646 (+ 78us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.555707 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.555793 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260923 22:08:01.556360 3735 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40349
I20260923 22:08:01.556445 3735 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:01.556581 3735 heartbeater.cc:511] Master 127.1.161.190:40349 requested a full tablet report, sending...
I20260923 22:08:01.556903 3392 ts_manager.cc:194] Registered new tserver with Master: cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:01.557353 3392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:45697
I20260923 22:08:01.560920 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 3604
I20260923 22:08:01.561013 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:01.561160 1670 external_mini_cluster.cc:1428] Setting key ca3de029ac446e2a44518ee652e6d4c8
I20260923 22:08:01.561828 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.557207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.130:37835)
0923 22:08:01.557298 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.557306 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.557355 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.557487 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.557487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.557654 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.557710 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.557711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.558283 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.558283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.558722 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.558723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.559052 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.559052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.559119 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.559520 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.559524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.559765 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.560901 (+ 1136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.560902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.560902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.561074 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.561238 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.561239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.561239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.561287 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.561575 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.561575 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.561614 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.561679 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.561760 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":50,"threads_started":1}
I20260923 22:08:01.561862 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.557256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:37835 (local address 127.1.161.190:40349)
0923 22:08:01.557293 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.557295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.557343 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:08:01.557402 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.557403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.557410 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.557456 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.557785 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.558199 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.558794 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.558988 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.559052 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.559851 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.559852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.559853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.559859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.560823 (+ 964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.561130 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.561131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.561131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.561180 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.561337 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.561338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.561338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.561511 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.561562 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.561709 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.561773 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:01.581665 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40349
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:01.651710 3744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:01.651870 3744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:01.651891 3744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:01.652910 3744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:01.652944 3744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:01.652961 3744 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 22:08:01.653000 3744 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:01.653023 3744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:01.654212 3744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:01.654402 3744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:01.654610 3744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:01.659049 3744 server_base.cc:1034] running on GCE node
W20260923 22:08:01.659060 3749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.658982 3750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:01.658986 3752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:01.659410 3744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:01.659698 3744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:01.660853 3744 hybrid_clock.cc:648] HybridClock initialized: now 1790201281660821 us; error 49 us; skew 500 ppm
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:01.663302 3744 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:01.663630 3744 webserver.cc:460] Webserver started at http://127.1.161.131:37897/ using document root <none> and password file <none>
I20260923 22:08:01.663789 3744 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:01.663825 3744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:01.663901 3744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:01.664708 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "e3e73b8c6373400abcfaf68ea9238ea1"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "2a7f4b0125a48221ead5589f4c602090"
server_key_iv: "e0c7330ce4a43936e765886892b924c6"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.664979 3744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "e3e73b8c6373400abcfaf68ea9238ea1"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "2a7f4b0125a48221ead5589f4c602090"
server_key_iv: "e0c7330ce4a43936e765886892b924c6"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.666155 3744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:01.666739 3759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.667065 3744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.667152 3744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "e3e73b8c6373400abcfaf68ea9238ea1"
format_stamp: "Formatted at 2026-09-23 22:08:01 on dist-test-slave-npvd"
server_key: "2a7f4b0125a48221ead5589f4c602090"
server_key_iv: "e0c7330ce4a43936e765886892b924c6"
server_key_version: "encryptionkey@0"
I20260923 22:08:01.667258 3744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:01.682274 3744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:01.684064 3744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:01.684155 3744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:01.684504 3744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:01.684535 3744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.684556 3744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:01.684589 3744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.690610 3744 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:44941
I20260923 22:08:01.690658 3874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:44941 every 8 connection(s)
I20260923 22:08:01.690935 3744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.694106 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.684976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:57169 (local address 127.1.161.190:40349)
0923 22:08:01.685042 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.685044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.685258 (+ 214us) server_negotiation.cc:407] Connection header received
0923 22:08:01.685667 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.685668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.685680 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.685756 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.686460 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.687144 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.688547 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.688810 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.688876 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.691522 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.691523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.691524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.691531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.692815 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.693202 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.693203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.693204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.693258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.693506 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.693507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.693507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.693611 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.693682 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.693927 (+ 245us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.694015 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:08:01.694213 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.685000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.131:57169)
0923 22:08:01.685137 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.685171 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.685582 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.685813 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.685817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.686131 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.686364 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.686370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.687245 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.687246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.688449 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.688452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.688901 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.688902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.689049 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.689616 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.689625 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.691413 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.692910 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.692915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.692917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.693115 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.693319 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.693320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.693321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.693424 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.693674 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.693679 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.693806 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.693905 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.694031 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260923 22:08:01.694655 3875 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40349
I20260923 22:08:01.694728 3875 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:01.694875 3875 heartbeater.cc:511] Master 127.1.161.190:40349 requested a full tablet report, sending...
I20260923 22:08:01.695258 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 3744
I20260923 22:08:01.695299 3391 ts_manager.cc:194] Registered new tserver with Master: e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941)
I20260923 22:08:01.695346 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:01.695544 1670 external_mini_cluster.cc:1428] Setting key 0055612b0f8ea80bc0ff72b5664a0aba
I20260923 22:08:01.695729 3391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:57169
I20260923 22:08:01.696676 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:01.700975 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.695616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.1.161.131:49609)
0923 22:08:01.695719 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.695728 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.695772 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.695898 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.695899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.696084 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.696133 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.696135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.696771 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.696771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.697410 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.697413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.697697 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.697698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.697767 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.698078 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.698082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.698412 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.699981 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.699981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.699982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.700150 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.700360 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.700361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.700361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.700434 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.700674 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.700674 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.700710 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.700782 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.700885 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":62,"threads_started":1}
I20260923 22:08:01.700963 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.695759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:49609 (local address 127.1.161.190:40349)
0923 22:08:01.695791 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.695793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.695801 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.695819 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.695819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.695826 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.695865 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.696204 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.696680 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.697483 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.697629 (+ 146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.697683 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.698505 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.698506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.698507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.698514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.699880 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.700238 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.700239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.700239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.700289 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.700519 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.700519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.700520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.700611 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.700669 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.700802 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.700885 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.716089 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.710578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.1.161.190:40349)
0923 22:08:01.710613 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.710615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.710640 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:08:01.710683 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.710684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.710690 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.710739 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.711235 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.711619 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.712231 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.712378 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.712419 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.714185 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.714186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.714186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.714191 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.715236 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.715569 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.715570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.715570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.715618 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.715779 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.715779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.715779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.715858 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.715923 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.715992 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.716040 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:08:01.717357 3391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:01.718784 3391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:01.724412 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.720827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35881 (local address 127.0.0.1:40514)
0923 22:08:01.720929 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.720962 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.721027 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.721264 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.721267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.721269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:01.721326 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.721328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722234 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.722235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.723295 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.723297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724142 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.724143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.724220 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.724231 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.724296 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.724339 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":62,"threads_started":1}
I20260923 22:08:01.724454 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.720902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.1.161.130:35881)
0923 22:08:01.721051 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.721056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.721067 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:01.721102 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.721104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.721118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.721196 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:01.721388 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722163 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.723377 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724063 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.724130 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.724189 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.724318 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.724353 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260923 22:08:01.724692 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.721040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:44941 (local address 127.0.0.1:34556)
0923 22:08:01.721164 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.721172 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.721210 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.721592 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.721592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.721594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:01.721637 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.721638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722705 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.722705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.723624 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.723626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724471 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.724472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.724534 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.724538 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.724568 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.724611 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":29,"thread_start_us":68,"threads_started":1}
I20260923 22:08:01.724876 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.721189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.0.0.1:39764)
0923 22:08:01.721323 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.721330 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.721361 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.721744 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.721745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.721746 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:01.721778 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.721779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722859 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.722860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.723742 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.723744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724449 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.724450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.724508 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.724513 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.724541 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.724575 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":48,"threads_started":1}
I20260923 22:08:01.724871 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.721351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.1.161.129:42749)
0923 22:08:01.721500 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.721503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.721511 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.721541 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.721543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.721555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.721835 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:01.721889 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722795 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.723812 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724370 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.724650 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.724707 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.724727 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.724756 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260923 22:08:01.725244 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.721134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.1.161.131:44941)
0923 22:08:01.721272 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.721276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.721285 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:01.721495 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.721498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.721511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.721587 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:01.721708 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.722637 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.723720 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.724375 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.725059 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.725107 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.725129 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.725163 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260923 22:08:01.725199 3669 tablet_service.cc:1467] Processing CreateTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 (DEFAULT_TABLE table=test-table [id=824f03e647e446549ce4f585e4e4158e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.725484 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 337cf2b7ced94df4a8cf8376fd32fa48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.727888 3895 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Bootstrap starting.
I20260923 22:08:01.728293 3809 tablet_service.cc:1467] Processing CreateTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 (DEFAULT_TABLE table=test-table [id=824f03e647e446549ce4f585e4e4158e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.728477 3895 tablet_bootstrap.cc:654] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.728556 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 337cf2b7ced94df4a8cf8376fd32fa48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.728605 3529 tablet_service.cc:1467] Processing CreateTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 (DEFAULT_TABLE table=test-table [id=824f03e647e446549ce4f585e4e4158e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.728760 3895 log.cc:824] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.728868 3529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 337cf2b7ced94df4a8cf8376fd32fa48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.731608 3897 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Bootstrap starting.
I20260923 22:08:01.731738 3895 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: No bootstrap required, opened a new log
I20260923 22:08:01.731849 3895 ts_tablet_manager.cc:1397] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 22:08:01.732452 3897 tablet_bootstrap.cc:654] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.732724 3897 log.cc:824] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.732990 3898 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Bootstrap starting.
I20260923 22:08:01.733351 3897 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: No bootstrap required, opened a new log
I20260923 22:08:01.733410 3897 ts_tablet_manager.cc:1397] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:01.733348 3895 raft_consensus.cc:348] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.733493 3895 raft_consensus.cc:374] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.733528 3898 tablet_bootstrap.cc:654] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.733527 3895 raft_consensus.cc:729] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5cffbaec4467c817942dcdf60a248, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.733633 3895 consensus_queue.cc:260] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.733807 3898 log.cc:824] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:01.733873 3895 ts_tablet_manager.cc:1428] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:01.733911 3735 heartbeater.cc:503] Master 127.1.161.190:40349 was elected leader, sending a full tablet report...
I20260923 22:08:01.734581 3898 tablet_bootstrap.cc:492] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: No bootstrap required, opened a new log
I20260923 22:08:01.734648 3898 ts_tablet_manager.cc:1397] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:01.735316 3897 raft_consensus.cc:348] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.735453 3897 raft_consensus.cc:374] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.735493 3897 raft_consensus.cc:729] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.735608 3897 consensus_queue.cc:260] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.735822 3897 ts_tablet_manager.cc:1428] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:08:01.735847 3875 heartbeater.cc:503] Master 127.1.161.190:40349 was elected leader, sending a full tablet report...
I20260923 22:08:01.736449 3898 raft_consensus.cc:348] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.736588 3898 raft_consensus.cc:374] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.736616 3898 raft_consensus.cc:729] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.736708 3898 consensus_queue.cc:260] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.736958 3898 ts_tablet_manager.cc:1428] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:01.736994 3595 heartbeater.cc:503] Master 127.1.161.190:40349 was elected leader, sending a full tablet report...
I20260923 22:08:01.769991 3903 raft_consensus.cc:482] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:01.770079 3903 raft_consensus.cc:504] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.770365 3903 leader_election.cc:288] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881), e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941)
I20260923 22:08:01.773008 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.770591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:36921 (local address 127.1.161.130:35881)
0923 22:08:01.770661 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.770663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.770671 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.770789 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.770790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.770799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.770847 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:01.771009 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.771485 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772346 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.772682 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772730 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.772736 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.772922 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.772954 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 22:08:01.773041 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.770749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:59015 (local address 127.1.161.131:44941)
0923 22:08:01.770790 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.770791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.770887 (+ 96us) server_negotiation.cc:407] Connection header received
0923 22:08:01.770914 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.770915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.770924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.770999 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:01.771126 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.771601 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772406 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.772728 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772791 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.772798 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.772955 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.772986 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:01.773044 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.770549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35881 (local address 127.1.161.129:36921)
0923 22:08:01.770596 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.770636 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.770685 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.770884 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.770885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.770887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:01.770950 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.770951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.771555 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.771556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.772270 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.772272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.772758 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.772861 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.772864 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.772909 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.772967 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:08:01.773054 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.770681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:44941 (local address 127.1.161.129:59015)
0923 22:08:01.770828 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.770837 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.770882 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.771005 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.771006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.771007 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:01.771065 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.771066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.771674 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.771675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.772301 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.772322 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.772813 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.772814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.772875 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.772878 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.772938 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.772969 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260923 22:08:01.773331 3829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "337cf2b7ced94df4a8cf8376fd32fa48" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" is_pre_election: true
I20260923 22:08:01.773367 3689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "337cf2b7ced94df4a8cf8376fd32fa48" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248" is_pre_election: true
I20260923 22:08:01.773451 3829 raft_consensus.cc:2454] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 0.
I20260923 22:08:01.773473 3689 raft_consensus.cc:2454] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 0.
I20260923 22:08:01.773658 3483 leader_election.cc:302] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd5cffbaec4467c817942dcdf60a248, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:01.773767 3903 raft_consensus.cc:2781] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:01.773821 3903 raft_consensus.cc:482] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:01.773846 3903 raft_consensus.cc:3037] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.774381 3903 raft_consensus.cc:504] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.774503 3903 leader_election.cc:288] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 election: Requested vote from peers cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881), e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941)
I20260923 22:08:01.774636 3689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "337cf2b7ced94df4a8cf8376fd32fa48" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248"
I20260923 22:08:01.774646 3829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "337cf2b7ced94df4a8cf8376fd32fa48" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e73b8c6373400abcfaf68ea9238ea1"
I20260923 22:08:01.774699 3829 raft_consensus.cc:3037] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.774691 3689 raft_consensus.cc:3037] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.775279 3829 raft_consensus.cc:2454] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 1.
I20260923 22:08:01.775331 3689 raft_consensus.cc:2454] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 1.
I20260923 22:08:01.775473 3483 leader_election.cc:302] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:01.775558 3903 raft_consensus.cc:2781] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.775668 3903 raft_consensus.cc:686] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 LEADER]: Becoming Leader. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Running, Role: LEADER
I20260923 22:08:01.775764 3903 consensus_queue.cc:237] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } }
I20260923 22:08:01.776373 3392 catalog_manager.cc:5495] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } health_report { overall_health: HEALTHY } } }
Sep 23 22:08:01 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201281, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:01.804898 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.799696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40476 (local address 127.1.161.190:40349)
0923 22:08:01.799753 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.799755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.799843 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:08:01.799885 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.799886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.799894 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.799944 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:01.800322 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.800733 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.801260 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.801400 (+ 140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.801438 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.803128 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:01.803130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:01.803130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:01.803134 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:01.804128 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.804403 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.804404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.804404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.804443 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:01.804589 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:01.804590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:01.804590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:01.804664 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:01.804705 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.804798 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.804849 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 22:08:01.804948 3736 tablet.cc:2369] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:01.805043 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.799628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40349 (local address 127.0.0.1:40476)
0923 22:08:01.799782 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.799790 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.799839 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.799965 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.799965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.800221 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:01.800263 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.800264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.800788 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.800788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.801204 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.801205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.801448 (+ 243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.801449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.801490 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.801942 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:01.801945 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:01.803066 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:01.804186 (+ 1120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.804190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.804192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.804347 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.804491 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:01.804491 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:01.804492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:01.804545 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:01.804709 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:01.804714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:01.804744 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.804787 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.804925 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":55,"thread_start_us":88,"threads_started":1}
I20260923 22:08:01.806645 3392 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:40476:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:08:01.806819 3392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:01.807900 3529 tablet_service.cc:1467] Processing CreateTablet for tablet d4436a025f0343e782bb6dbab22fdb63 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62bdb4e652c740fcb6c9430110e995dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:01.807941 3809 tablet_service.cc:1467] Processing CreateTablet for tablet d4436a025f0343e782bb6dbab22fdb63 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62bdb4e652c740fcb6c9430110e995dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:01.807940 3669 tablet_service.cc:1467] Processing CreateTablet for tablet d4436a025f0343e782bb6dbab22fdb63 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62bdb4e652c740fcb6c9430110e995dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:01.808038 3529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4436a025f0343e782bb6dbab22fdb63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.808053 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4436a025f0343e782bb6dbab22fdb63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.808053 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4436a025f0343e782bb6dbab22fdb63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.809041 3898 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: Bootstrap starting.
I20260923 22:08:01.809193 3895 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: Bootstrap starting.
I20260923 22:08:01.809494 3898 tablet_bootstrap.cc:654] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.809883 3895 tablet_bootstrap.cc:654] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.809927 3897 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: Bootstrap starting.
I20260923 22:08:01.810488 3897 tablet_bootstrap.cc:654] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.810544 3895 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: No bootstrap required, opened a new log
I20260923 22:08:01.810593 3895 ts_tablet_manager.cc:1397] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.810742 3895 raft_consensus.cc:348] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.810858 3895 raft_consensus.cc:374] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.810892 3895 raft_consensus.cc:729] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5cffbaec4467c817942dcdf60a248, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.810937 3895 consensus_queue.cc:260] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.811022 3898 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: No bootstrap required, opened a new log
I20260923 22:08:01.811033 3895 ts_tablet_manager.cc:1428] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:01.811074 3898 ts_tablet_manager.cc:1397] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:01.811122 3900 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.811244 3900 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.811242 3898 raft_consensus.cc:348] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.811327 3898 raft_consensus.cc:374] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.811348 3898 raft_consensus.cc:729] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.811412 3898 consensus_queue.cc:260] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.811518 3898 ts_tablet_manager.cc:1428] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.811566 3903 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.811651 3903 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.811825 3897 tablet_bootstrap.cc:492] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: No bootstrap required, opened a new log
I20260923 22:08:01.811860 3897 ts_tablet_manager.cc:1397] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:01.812016 3897 raft_consensus.cc:348] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.812076 3897 raft_consensus.cc:374] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.812098 3897 raft_consensus.cc:729] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.812139 3897 consensus_queue.cc:260] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.812213 3897 ts_tablet_manager.cc:1428] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.812264 3902 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.812357 3902 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.875725 3903 raft_consensus.cc:482] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:01.875778 3903 raft_consensus.cc:504] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.875900 3903 leader_election.cc:288] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881), e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941)
I20260923 22:08:01.876113 3829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4436a025f0343e782bb6dbab22fdb63" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" is_pre_election: true
I20260923 22:08:01.876103 3689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4436a025f0343e782bb6dbab22fdb63" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248" is_pre_election: true
I20260923 22:08:01.876181 3689 raft_consensus.cc:2454] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 0.
I20260923 22:08:01.876181 3829 raft_consensus.cc:2454] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 0.
I20260923 22:08:01.876289 3483 leader_election.cc:302] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd5cffbaec4467c817942dcdf60a248, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:01.876364 3903 raft_consensus.cc:2781] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:01.876392 3903 raft_consensus.cc:482] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:01.876415 3903 raft_consensus.cc:3037] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.876902 3903 raft_consensus.cc:504] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.877003 3903 leader_election.cc:288] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 election: Requested vote from peers cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881), e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941)
I20260923 22:08:01.877173 3829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4436a025f0343e782bb6dbab22fdb63" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e73b8c6373400abcfaf68ea9238ea1"
I20260923 22:08:01.877171 3689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4436a025f0343e782bb6dbab22fdb63" candidate_uuid: "fcb10a317ad34e5fb5c436b5b250e190" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248"
I20260923 22:08:01.877235 3829 raft_consensus.cc:3037] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.877235 3689 raft_consensus.cc:3037] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:01.877854 3829 raft_consensus.cc:2454] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 1.
I20260923 22:08:01.877853 3689 raft_consensus.cc:2454] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb10a317ad34e5fb5c436b5b250e190 in term 1.
I20260923 22:08:01.877996 3483 leader_election.cc:302] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:01.878091 3903 raft_consensus.cc:2781] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:01.878139 3903 raft_consensus.cc:686] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 LEADER]: Becoming Leader. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Running, Role: LEADER
I20260923 22:08:01.878198 3903 consensus_queue.cc:237] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.878458 3905 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: TxnStatusTablet state changed. Reason: New leader fcb10a317ad34e5fb5c436b5b250e190. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.878567 3905 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:01.878656 3391 catalog_manager.cc:5495] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:01.878755 3917 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:08:01.878846 3917 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:08:01.883764 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.881728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.1.161.129:42749)
0923 22:08:01.881766 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.881768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.881776 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.881845 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.881846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.881860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.881908 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:01.881998 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.882391 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.883015 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.883246 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.883321 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.883484 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:01.883571 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:01.883645 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.883691 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.883708 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:01.883780 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.881669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.0.0.1:39770)
0923 22:08:01.881730 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.881739 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.881781 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.881908 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.881909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.881910 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:01.881949 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.881951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.882459 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.882460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.882941 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.882942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.883323 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.883324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.883381 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.883399 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:01.883637 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:01.883644 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.883679 (+ 35us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.883724 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:08:01.885301 3689 raft_consensus.cc:1261] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Refusing update from remote peer fcb10a317ad34e5fb5c436b5b250e190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:01.885301 3829 raft_consensus.cc:1261] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Refusing update from remote peer fcb10a317ad34e5fb5c436b5b250e190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:01.885524 3903 consensus_queue.cc:1035] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.885613 3905 consensus_queue.cc:1035] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.886560 3900 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: TxnStatusTablet state changed. Reason: New leader fcb10a317ad34e5fb5c436b5b250e190. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.886646 3900 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.886600 3902 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: TxnStatusTablet state changed. Reason: New leader fcb10a317ad34e5fb5c436b5b250e190. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.886682 3902 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.887759 3905 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.887835 3905 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:01.887785 3925 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.887863 3925 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P fcb10a317ad34e5fb5c436b5b250e190: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:01.887984 3920 mvcc.cc:205] Tried to move back new op lower bound from 7332664450599649280 to 7332664450572800000. Current Snapshot: MvccSnapshot[applied={T|T < 7332664450599649280}]
I20260923 22:08:01.887967 3900 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.888041 3900 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P cbd5cffbaec4467c817942dcdf60a248: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.888437 3902 tablet_replica.cc:439] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcb10a317ad34e5fb5c436b5b250e190" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } } }
I20260923 22:08:01.888507 3902 tablet_replica.cc:442] T d4436a025f0343e782bb6dbab22fdb63 P e3e73b8c6373400abcfaf68ea9238ea1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:01.888796 3921 mvcc.cc:205] Tried to move back new op lower bound from 7332664450599649280 to 7332664450572800000. Current Snapshot: MvccSnapshot[applied={T|T < 7332664450599649280}]
I20260923 22:08:01.893332 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.890711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39774 (local address 127.1.161.129:42749)
0923 22:08:01.890759 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.890761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.890769 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:01.890830 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.890831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.890841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.890893 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:01.890988 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.891588 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.892569 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.892871 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.892938 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.893094 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:01.893123 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:01.893171 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.893227 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.893244 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:01.899540 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.897507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:37109 (local address 127.1.161.129:42749)
0923 22:08:01.897534 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:01.897536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:01.897542 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:01.897597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:01.897597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:01.897603 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:01.897642 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:01.897740 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.898071 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.898928 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.899067 (+ 139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.899117 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.899333 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:01.899369 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:01.899404 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:01.899455 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:08:01.899474 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":13}
I20260923 22:08:01.899603 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:01.897462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.1.161.129:37109)
0923 22:08:01.897514 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:01.897522 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:01.897551 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:01.897645 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:01.897646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:01.897647 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:01.897694 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.897695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.898135 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.898135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.898858 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:01.898860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:01.899137 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:01.899138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:01.899221 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:01.899234 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:01.899406 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:01.899407 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:01.899454 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:01.899490 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:08:01.901329 3829 raft_consensus.cc:1261] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Refusing update from remote peer fcb10a317ad34e5fb5c436b5b250e190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:01.901335 3689 raft_consensus.cc:1261] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Refusing update from remote peer fcb10a317ad34e5fb5c436b5b250e190: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:01.901499 3903 consensus_queue.cc:1035] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.901624 3905 consensus_queue.cc:1035] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:01.903120 3922 mvcc.cc:205] Tried to move back new op lower bound from 7332664450666237952 to 7332664450153582592. Current Snapshot: MvccSnapshot[applied={T|T < 7332664450666237952}]
I20260923 22:08:01.912717 3392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:01.912793 3392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:01.913335 3392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Sending DeleteTablet for 3 replicas of tablet 337cf2b7ced94df4a8cf8376fd32fa48
I20260923 22:08:01.913555 3669 tablet_service.cc:1514] Processing DeleteTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:40514
I20260923 22:08:01.913596 3809 tablet_service.cc:1514] Processing DeleteTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:34556
I20260923 22:08:01.913662 3528 tablet_service.cc:1514] Processing DeleteTablet for tablet 337cf2b7ced94df4a8cf8376fd32fa48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:01 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260923 22:08:01.914019 3392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:01.914143 3392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:01.914157 3937 tablet_replica.cc:330] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: stopping tablet replica
I20260923 22:08:01.914271 3937 raft_consensus.cc:2229] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:01.914311 3937 raft_consensus.cc:2258] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:01.914639 3937 ts_tablet_manager.cc:1905] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:01.915120 3669 tablet_service.cc:1467] Processing CreateTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (DEFAULT_TABLE table=test-table [id=aa305b0b992f401197dd8c364e830785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.915141 3809 tablet_service.cc:1467] Processing CreateTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (DEFAULT_TABLE table=test-table [id=aa305b0b992f401197dd8c364e830785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.915269 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.915282 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.916126 3937 ts_tablet_manager.cc:1918] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:01.916173 3937 log.cc:1196] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/337cf2b7ced94df4a8cf8376fd32fa48
I20260923 22:08:01.916306 3897 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Bootstrap starting.
I20260923 22:08:01.916402 3895 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Bootstrap starting.
I20260923 22:08:01.916392 3937 ts_tablet_manager.cc:1939] T 337cf2b7ced94df4a8cf8376fd32fa48 P cbd5cffbaec4467c817942dcdf60a248: Deleting consensus metadata
I20260923 22:08:01.916617 3897 tablet_bootstrap.cc:654] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.916744 3895 tablet_bootstrap.cc:654] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.917008 3379 catalog_manager.cc:4841] TS cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881): tablet 337cf2b7ced94df4a8cf8376fd32fa48 (table test-table [id=824f03e647e446549ce4f585e4e4158e]) successfully deleted
I20260923 22:08:01.917137 3897 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: No bootstrap required, opened a new log
I20260923 22:08:01.917171 3897 ts_tablet_manager.cc:1397] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:01.917171 3528 tablet_service.cc:1467] Processing CreateTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (DEFAULT_TABLE table=test-table [id=aa305b0b992f401197dd8c364e830785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:01.917255 3528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:01.917284 3897 raft_consensus.cc:348] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.917341 3897 raft_consensus.cc:374] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.917363 3897 raft_consensus.cc:729] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.917400 3897 consensus_queue.cc:260] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.917471 3897 ts_tablet_manager.cc:1428] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.917953 3895 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: No bootstrap required, opened a new log
I20260923 22:08:01.918002 3895 ts_tablet_manager.cc:1397] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:01.918119 3895 raft_consensus.cc:348] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.918179 3895 raft_consensus.cc:374] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.918202 3895 raft_consensus.cc:729] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5cffbaec4467c817942dcdf60a248, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.918238 3895 consensus_queue.cc:260] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.918315 3895 ts_tablet_manager.cc:1428] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.918363 3898 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Bootstrap starting.
I20260923 22:08:01.918690 3898 tablet_bootstrap.cc:654] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:01.919212 3935 tablet_replica.cc:330] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: stopping tablet replica
I20260923 22:08:01.919224 3898 tablet_bootstrap.cc:492] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: No bootstrap required, opened a new log
I20260923 22:08:01.919250 3936 tablet_replica.cc:330] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: stopping tablet replica
I20260923 22:08:01.919265 3898 ts_tablet_manager.cc:1397] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:01.919313 3935 raft_consensus.cc:2229] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:01.919340 3936 raft_consensus.cc:2229] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:01.919375 3935 raft_consensus.cc:2258] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:01.919440 3936 raft_consensus.cc:2258] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:01.919420 3898 raft_consensus.cc:348] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.919474 3898 raft_consensus.cc:374] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:01.919499 3898 raft_consensus.cc:729] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Initialized, Role: FOLLOWER
I20260923 22:08:01.919538 3898 consensus_queue.cc:260] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:01.919626 3898 ts_tablet_manager.cc:1428] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:01.920120 3936 ts_tablet_manager.cc:1905] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:01.920351 3935 ts_tablet_manager.cc:1905] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:01.921332 3936 ts_tablet_manager.cc:1918] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:01.921404 3936 log.cc:1196] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/337cf2b7ced94df4a8cf8376fd32fa48
I20260923 22:08:01.921604 3936 ts_tablet_manager.cc:1939] T 337cf2b7ced94df4a8cf8376fd32fa48 P fcb10a317ad34e5fb5c436b5b250e190: Deleting consensus metadata
I20260923 22:08:01.921985 3379 catalog_manager.cc:4841] TS fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749): tablet 337cf2b7ced94df4a8cf8376fd32fa48 (table test-table [id=824f03e647e446549ce4f585e4e4158e]) successfully deleted
I20260923 22:08:01.922049 3935 ts_tablet_manager.cc:1918] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:01.922114 3935 log.cc:1196] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/337cf2b7ced94df4a8cf8376fd32fa48
I20260923 22:08:01.922371 3935 ts_tablet_manager.cc:1939] T 337cf2b7ced94df4a8cf8376fd32fa48 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting consensus metadata
I20260923 22:08:01.922737 3379 catalog_manager.cc:4841] TS e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941): tablet 337cf2b7ced94df4a8cf8376fd32fa48 (table test-table [id=824f03e647e446549ce4f585e4e4158e]) successfully deleted
W20260923 22:08:01.941417 3876 tablet.cc:2369] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:02.030205 3902 raft_consensus.cc:482] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:02.030305 3902 raft_consensus.cc:504] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:02.030670 3902 leader_election.cc:288] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:02.034044 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.030928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60463 (local address 127.1.161.129:42749)
0923 22:08:02.030971 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:02.030973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:02.030983 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:02.031078 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:02.031079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:02.031090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:02.031153 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:02.031365 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.032010 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.033161 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.033690 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.033755 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.033766 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:02.033940 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 22:08:02.033978 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:02.034056 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.030864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.1.161.131:60463)
0923 22:08:02.030917 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:02.030929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:02.030976 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:02.031215 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:02.031216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:02.031217 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:02.031285 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.031287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.032090 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.032091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.033046 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.033049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.033787 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.033788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.033864 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.033867 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:02.033933 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:08:02.033971 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:08:02.034329 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b3ce390f2d40dbbc5c5d6152c0af5a" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190" is_pre_election: true
I20260923 22:08:02.034439 3549 raft_consensus.cc:2454] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:02.034636 3763 leader_election.cc:302] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:02.034767 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.031096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60225 (local address 127.1.161.130:35881)
0923 22:08:02.031134 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:02.031136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:02.031193 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:08:02.031221 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:02.031222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:02.031231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:02.031283 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:02.031766 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.032497 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.033785 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.034366 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.034433 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.034440 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:02.034651 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 22:08:02.034698 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:02.034750 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.030984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35881 (local address 127.1.161.131:60225)
0923 22:08:02.031082 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:02.031091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:02.031130 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:02.031300 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:02.031301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:02.031303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:02.031679 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.031682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.032585 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.032586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.033679 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.033681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.034488 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.034489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.034566 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.034570 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:02.034626 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:08:02.034665 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260923 22:08:02.034751 3902 raft_consensus.cc:2781] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:02.034827 3902 raft_consensus.cc:482] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:02.034853 3902 raft_consensus.cc:3037] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:02.035609 3902 raft_consensus.cc:504] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:02.035825 3902 leader_election.cc:288] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Requested vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:02.036077 3689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b3ce390f2d40dbbc5c5d6152c0af5a" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248" is_pre_election: true
I20260923 22:08:02.036177 3689 raft_consensus.cc:2454] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:02.036100 3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b3ce390f2d40dbbc5c5d6152c0af5a" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248"
I20260923 22:08:02.036190 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b3ce390f2d40dbbc5c5d6152c0af5a" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190"
I20260923 22:08:02.036250 3688 raft_consensus.cc:3037] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:02.036257 3549 raft_consensus.cc:3037] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:02.037047 3549 raft_consensus.cc:2454] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:02.037237 3763 leader_election.cc:302] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:02.037245 3688 raft_consensus.cc:2454] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:02.037336 3902 raft_consensus.cc:2781] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:02.037433 3902 raft_consensus.cc:686] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Becoming Leader. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Running, Role: LEADER
I20260923 22:08:02.037504 3902 consensus_queue.cc:237] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:02.038193 3392 catalog_manager.cc:5495] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:02.067406 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.065176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.1.161.131:44941)
0923 22:08:02.065209 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:02.065211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:02.065242 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:02.065270 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:02.065270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:02.065287 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:02.065355 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:02.065463 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.066052 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.066652 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.066926 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.066990 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.067094 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:02.067232 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:02.067310 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:02.067326 (+ 16us) server_negotiation.cc:299] Negotiation successful
0923 22:08:02.067344 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:02.072016 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.069690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:50815 (local address 127.1.161.129:42749)
0923 22:08:02.069729 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:02.069731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:02.069760 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:08:02.069842 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:02.069842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:02.069851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:02.069911 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:02.070029 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.070451 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.071457 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.071622 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.071669 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.071840 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:02.071863 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:02.071897 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:02.071948 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:08:02.071960 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:02.072052 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.069627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.1.161.131:50815)
0923 22:08:02.069715 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:02.069726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:02.069772 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:02.069913 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:02.069913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:02.069915 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:02.069970 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.069972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.070544 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.070545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.071386 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.071389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.071694 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.071695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.071754 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.071767 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:02.071901 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:02.071902 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:02.071947 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:02.071979 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260923 22:08:02.073648 3688 raft_consensus.cc:1261] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:02.073781 3549 raft_consensus.cc:1261] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:02.073879 3942 consensus_queue.cc:1035] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:02.073974 3902 consensus_queue.cc:1035] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:02.080500 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.078255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:45485 (local address 127.1.161.131:44941)
0923 22:08:02.078290 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:02.078292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:02.078298 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:02.078315 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:02.078315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:02.078323 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:02.078369 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:02.078487 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.078895 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.079863 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.080106 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.080156 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.080319 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:02.080340 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:02.080376 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:02.080447 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 22:08:02.080460 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:02.080549 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:02.078080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:44941 (local address 127.1.161.129:45485)
0923 22:08:02.078149 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:02.078157 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:02.078201 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:02.078390 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:02.078391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:02.078392 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:02.078432 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.078433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.078968 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.078969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.079766 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:02.079768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:02.080184 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:02.080185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:02.080241 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:02.080246 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:02.080394 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:02.080396 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:02.080446 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:02.080483 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 22:08:02.080824 3922 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:08:03.598520 3391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:03.598608 3391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:03.599364 3391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Sending DeleteTablet for 3 replicas of tablet c6b3ce390f2d40dbbc5c5d6152c0af5a
I20260923 22:08:03.599576 3529 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260923 22:08:03.599650 3669 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:40514
I20260923 22:08:03.599802 3809 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b3ce390f2d40dbbc5c5d6152c0af5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:34556
I20260923 22:08:03.600018 3972 tablet_replica.cc:330] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: stopping tablet replica
I20260923 22:08:03.600039 3973 tablet_replica.cc:330] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: stopping tablet replica
I20260923 22:08:03.600097 3973 raft_consensus.cc:2229] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:03.600106 3971 tablet_replica.cc:330] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: stopping tablet replica
I20260923 22:08:03.600142 3972 raft_consensus.cc:2229] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:03.600160 3971 raft_consensus.cc:2229] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:03.600191 3971 raft_consensus.cc:2258] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.600210 3972 raft_consensus.cc:2258] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.600283 3973 raft_consensus.cc:2258] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.600487 3971 ts_tablet_manager.cc:1905] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.600497 3972 ts_tablet_manager.cc:1905] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.600551 3973 ts_tablet_manager.cc:1905] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.602276 3971 ts_tablet_manager.cc:1918] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:08:03.602288 3972 ts_tablet_manager.cc:1918] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:08:03.602288 3973 ts_tablet_manager.cc:1918] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:08:03.602331 3973 log.cc:1196] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/c6b3ce390f2d40dbbc5c5d6152c0af5a
I20260923 22:08:03.602331 3972 log.cc:1196] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/c6b3ce390f2d40dbbc5c5d6152c0af5a
I20260923 22:08:03.602449 3971 log.cc:1196] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/c6b3ce390f2d40dbbc5c5d6152c0af5a
I20260923 22:08:03.602545 3972 ts_tablet_manager.cc:1939] T c6b3ce390f2d40dbbc5c5d6152c0af5a P cbd5cffbaec4467c817942dcdf60a248: Deleting consensus metadata
I20260923 22:08:03.602545 3973 ts_tablet_manager.cc:1939] T c6b3ce390f2d40dbbc5c5d6152c0af5a P e3e73b8c6373400abcfaf68ea9238ea1: Deleting consensus metadata
I20260923 22:08:03.602720 3971 ts_tablet_manager.cc:1939] T c6b3ce390f2d40dbbc5c5d6152c0af5a P fcb10a317ad34e5fb5c436b5b250e190: Deleting consensus metadata
I20260923 22:08:03.602813 3379 catalog_manager.cc:4841] TS cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881): tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (table test-table [id=aa305b0b992f401197dd8c364e830785]) successfully deleted
I20260923 22:08:03.602890 3379 catalog_manager.cc:4841] TS e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941): tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (table test-table [id=aa305b0b992f401197dd8c364e830785]) successfully deleted
I20260923 22:08:03.603024 3379 catalog_manager.cc:4841] TS fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749): tablet c6b3ce390f2d40dbbc5c5d6152c0af5a (table test-table [id=aa305b0b992f401197dd8c364e830785]) successfully deleted
Sep 23 22:08:03 dist-test-slave-npvd krb5kdc[3347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:08:03 dist-test-slave-npvd krb5kdc[3347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:03.621061 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:03.614984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.1.161.190:40349)
0923 22:08:03.615091 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:03.615092 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:03.615099 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:03.615115 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:03.615116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:03.615123 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:03.615214 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:03.615648 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.616085 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.616703 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.616865 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.616917 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:03.618680 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:03.618681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:03.618681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:03.618686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:03.620140 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:03.620419 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:03.620420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:03.620420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:03.620483 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:03.620622 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:03.620622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:03.620623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:03.620708 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:03.620774 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:03.620853 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 22:08:03.620969 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":63,"threads_started":1}
I20260923 22:08:03.622373 3391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:03.622506 3391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:03.623607 3669 tablet_service.cc:1467] Processing CreateTablet for tablet 0c977d011aa84dba87766d506265ca84 (DEFAULT_TABLE table=test-table [id=a43ec16aa56d4201babe8b5a4a35e993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.623620 3809 tablet_service.cc:1467] Processing CreateTablet for tablet 0c977d011aa84dba87766d506265ca84 (DEFAULT_TABLE table=test-table [id=a43ec16aa56d4201babe8b5a4a35e993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.623633 3529 tablet_service.cc:1467] Processing CreateTablet for tablet 0c977d011aa84dba87766d506265ca84 (DEFAULT_TABLE table=test-table [id=a43ec16aa56d4201babe8b5a4a35e993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.623713 3529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c977d011aa84dba87766d506265ca84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.623709 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c977d011aa84dba87766d506265ca84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.623709 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c977d011aa84dba87766d506265ca84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.624531 3982 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Bootstrap starting.
I20260923 22:08:03.624887 3982 tablet_bootstrap.cc:654] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.625078 3983 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Bootstrap starting.
I20260923 22:08:03.625409 3983 tablet_bootstrap.cc:654] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.625453 3982 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: No bootstrap required, opened a new log
I20260923 22:08:03.625494 3982 ts_tablet_manager.cc:1397] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:03.625603 3982 raft_consensus.cc:348] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.625681 3982 raft_consensus.cc:374] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.625698 3984 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Bootstrap starting.
I20260923 22:08:03.625747 3982 raft_consensus.cc:729] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.625814 3982 consensus_queue.cc:260] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.625941 3983 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: No bootstrap required, opened a new log
I20260923 22:08:03.625995 3983 ts_tablet_manager.cc:1397] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:03.626034 3984 tablet_bootstrap.cc:654] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.626075 3982 ts_tablet_manager.cc:1428] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:03.626149 3983 raft_consensus.cc:348] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.626271 3983 raft_consensus.cc:374] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.626323 3983 raft_consensus.cc:729] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5cffbaec4467c817942dcdf60a248, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.626384 3983 consensus_queue.cc:260] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.626546 3983 ts_tablet_manager.cc:1428] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:03.626727 3984 tablet_bootstrap.cc:492] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: No bootstrap required, opened a new log
I20260923 22:08:03.626775 3984 ts_tablet_manager.cc:1397] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:03.626986 3984 raft_consensus.cc:348] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.627053 3984 raft_consensus.cc:374] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.627108 3984 raft_consensus.cc:729] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.627183 3984 consensus_queue.cc:260] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.627275 3984 ts_tablet_manager.cc:1428] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260923 22:08:03.670750 3596 tablet.cc:2369] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:03.678783 3948 raft_consensus.cc:482] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:03.678845 3948 raft_consensus.cc:504] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.678969 3948 leader_election.cc:288] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:03.679144 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c977d011aa84dba87766d506265ca84" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190" is_pre_election: true
I20260923 22:08:03.679278 3549 raft_consensus.cc:2454] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:03.679255 3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c977d011aa84dba87766d506265ca84" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248" is_pre_election: true
I20260923 22:08:03.679363 3688 raft_consensus.cc:2454] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:03.679459 3763 leader_election.cc:302] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:03.679558 3948 raft_consensus.cc:2781] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:03.679594 3948 raft_consensus.cc:482] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:03.679606 3948 raft_consensus.cc:3037] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.680284 3948 raft_consensus.cc:504] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.680379 3948 leader_election.cc:288] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Requested vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:03.680632 3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c977d011aa84dba87766d506265ca84" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248"
I20260923 22:08:03.680622 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c977d011aa84dba87766d506265ca84" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190"
I20260923 22:08:03.680694 3688 raft_consensus.cc:3037] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.680713 3549 raft_consensus.cc:3037] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.681288 3688 raft_consensus.cc:2454] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:03.681356 3549 raft_consensus.cc:2454] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:03.681464 3763 leader_election.cc:302] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbd5cffbaec4467c817942dcdf60a248, e3e73b8c6373400abcfaf68ea9238ea1; no voters:
I20260923 22:08:03.681555 3948 raft_consensus.cc:2781] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:03.681612 3948 raft_consensus.cc:686] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Becoming Leader. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Running, Role: LEADER
I20260923 22:08:03.681684 3948 consensus_queue.cc:237] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.682190 3392 catalog_manager.cc:5495] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:03.706995 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:03.703743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.1.161.131:44941)
0923 22:08:03.703880 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:03.703882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:03.703890 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:03.703914 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:03.703914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:03.703936 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:03.704005 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:03.704167 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.704695 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.705951 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:03.706255 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:03.706502 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:03.706639 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:03.706673 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:03.706714 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:03.706914 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:08:03.706933 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260923 22:08:03.709637 3688 raft_consensus.cc:1261] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:03.709668 3549 raft_consensus.cc:1261] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:03.709844 3948 consensus_queue.cc:1035] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:03.709981 3948 consensus_queue.cc:1035] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:03.719358 3392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:03.719441 3392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:03.720034 3392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Sending DeleteTablet for 3 replicas of tablet 0c977d011aa84dba87766d506265ca84
I20260923 22:08:03.720297 3529 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c977d011aa84dba87766d506265ca84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260923 22:08:03.720321 3809 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c977d011aa84dba87766d506265ca84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:34556
I20260923 22:08:03.720392 3971 tablet_replica.cc:330] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: stopping tablet replica
I20260923 22:08:03.720402 3973 tablet_replica.cc:330] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: stopping tablet replica
I20260923 22:08:03.720397 3669 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c977d011aa84dba87766d506265ca84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:03 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:40514
I20260923 22:08:03.720453 3973 raft_consensus.cc:2229] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:03.720446 3971 raft_consensus.cc:2229] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:03.720502 3973 raft_consensus.cc:2258] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.720504 3971 raft_consensus.cc:2258] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.720573 3972 tablet_replica.cc:330] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: stopping tablet replica
I20260923 22:08:03.720626 3972 raft_consensus.cc:2229] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:03.720662 3972 raft_consensus.cc:2258] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:03.720793 3973 ts_tablet_manager.cc:1905] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.720753 3392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:03.720890 3392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:03.720894 3971 ts_tablet_manager.cc:1905] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.721933 3809 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc7521bb04420e99d9b2df01e85efd (DEFAULT_TABLE table=test-table [id=9e58a4884c2a4732a76ddd3922620c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.721940 3669 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc7521bb04420e99d9b2df01e85efd (DEFAULT_TABLE table=test-table [id=9e58a4884c2a4732a76ddd3922620c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.722028 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc7521bb04420e99d9b2df01e85efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.722036 3809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc7521bb04420e99d9b2df01e85efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.722344 3973 ts_tablet_manager.cc:1918] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:03.722384 3973 log.cc:1196] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/0c977d011aa84dba87766d506265ca84
I20260923 22:08:03.722436 3971 ts_tablet_manager.cc:1918] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:03.722493 3971 log.cc:1196] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/0c977d011aa84dba87766d506265ca84
I20260923 22:08:03.722600 3973 ts_tablet_manager.cc:1939] T 0c977d011aa84dba87766d506265ca84 P e3e73b8c6373400abcfaf68ea9238ea1: Deleting consensus metadata
I20260923 22:08:03.722679 3971 ts_tablet_manager.cc:1939] T 0c977d011aa84dba87766d506265ca84 P fcb10a317ad34e5fb5c436b5b250e190: Deleting consensus metadata
I20260923 22:08:03.723146 3984 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Bootstrap starting.
I20260923 22:08:03.723307 3983 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: Bootstrap starting.
I20260923 22:08:03.723323 3972 ts_tablet_manager.cc:1905] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:03.723469 3984 tablet_bootstrap.cc:654] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.723497 3529 tablet_service.cc:1467] Processing CreateTablet for tablet 14fc7521bb04420e99d9b2df01e85efd (DEFAULT_TABLE table=test-table [id=9e58a4884c2a4732a76ddd3922620c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:03.723575 3983 tablet_bootstrap.cc:654] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.723579 3529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fc7521bb04420e99d9b2df01e85efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:03.724082 3983 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: No bootstrap required, opened a new log
I20260923 22:08:03.724131 3983 ts_tablet_manager.cc:1397] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:03.724246 3983 raft_consensus.cc:348] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.724326 3983 raft_consensus.cc:374] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.724344 3983 raft_consensus.cc:729] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd5cffbaec4467c817942dcdf60a248, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.724380 3983 consensus_queue.cc:260] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.724489 3983 ts_tablet_manager.cc:1428] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:03.724575 3972 ts_tablet_manager.cc:1918] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:03.724614 3972 log.cc:1196] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/0c977d011aa84dba87766d506265ca84
I20260923 22:08:03.724619 3982 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: Bootstrap starting.
I20260923 22:08:03.724727 3379 catalog_manager.cc:4841] TS fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749): tablet 0c977d011aa84dba87766d506265ca84 (table test-table [id=a43ec16aa56d4201babe8b5a4a35e993]) successfully deleted
I20260923 22:08:03.724793 3972 ts_tablet_manager.cc:1939] T 0c977d011aa84dba87766d506265ca84 P cbd5cffbaec4467c817942dcdf60a248: Deleting consensus metadata
I20260923 22:08:03.724866 3982 tablet_bootstrap.cc:654] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:03.724967 3379 catalog_manager.cc:4841] TS e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941): tablet 0c977d011aa84dba87766d506265ca84 (table test-table [id=a43ec16aa56d4201babe8b5a4a35e993]) successfully deleted
I20260923 22:08:03.725162 3984 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: No bootstrap required, opened a new log
I20260923 22:08:03.725224 3984 ts_tablet_manager.cc:1397] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:03.725325 3984 raft_consensus.cc:348] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.725379 3984 raft_consensus.cc:374] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.725397 3984 raft_consensus.cc:729] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.725412 3982 tablet_bootstrap.cc:492] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: No bootstrap required, opened a new log
I20260923 22:08:03.725445 3982 ts_tablet_manager.cc:1397] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:03.725430 3984 consensus_queue.cc:260] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.725522 3984 ts_tablet_manager.cc:1428] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:03.725523 3982 raft_consensus.cc:348] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.725574 3982 raft_consensus.cc:374] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:03.725595 3982 raft_consensus.cc:729] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb10a317ad34e5fb5c436b5b250e190, State: Initialized, Role: FOLLOWER
I20260923 22:08:03.725648 3982 consensus_queue.cc:260] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.725709 3379 catalog_manager.cc:4841] TS cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881): tablet 0c977d011aa84dba87766d506265ca84 (table test-table [id=a43ec16aa56d4201babe8b5a4a35e993]) successfully deleted
I20260923 22:08:03.725739 3982 ts_tablet_manager.cc:1428] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:03.787575 3962 raft_consensus.cc:482] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:03.787647 3962 raft_consensus.cc:504] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.787778 3962 leader_election.cc:288] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:03.787955 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc7521bb04420e99d9b2df01e85efd" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190" is_pre_election: true
I20260923 22:08:03.788036 3549 raft_consensus.cc:2454] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:03.788017 3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc7521bb04420e99d9b2df01e85efd" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248" is_pre_election: true
I20260923 22:08:03.788130 3688 raft_consensus.cc:2454] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 0.
I20260923 22:08:03.788177 3763 leader_election.cc:302] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:03.788290 3962 raft_consensus.cc:2781] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:03.788316 3962 raft_consensus.cc:482] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:03.788327 3962 raft_consensus.cc:3037] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.788853 3962 raft_consensus.cc:504] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.788956 3962 leader_election.cc:288] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Requested vote from peers fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749), cbd5cffbaec4467c817942dcdf60a248 (127.1.161.130:35881)
I20260923 22:08:03.789072 3549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc7521bb04420e99d9b2df01e85efd" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb10a317ad34e5fb5c436b5b250e190"
I20260923 22:08:03.789139 3549 raft_consensus.cc:3037] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.789105 3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fc7521bb04420e99d9b2df01e85efd" candidate_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd5cffbaec4467c817942dcdf60a248"
I20260923 22:08:03.789178 3688 raft_consensus.cc:3037] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:03.789654 3549 raft_consensus.cc:2454] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:03.789711 3688 raft_consensus.cc:2454] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e73b8c6373400abcfaf68ea9238ea1 in term 1.
I20260923 22:08:03.789840 3763 leader_election.cc:302] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3e73b8c6373400abcfaf68ea9238ea1, fcb10a317ad34e5fb5c436b5b250e190; no voters:
I20260923 22:08:03.789984 3962 raft_consensus.cc:2781] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:03.790043 3962 raft_consensus.cc:686] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Becoming Leader. State: Replica: e3e73b8c6373400abcfaf68ea9238ea1, State: Running, Role: LEADER
I20260923 22:08:03.790108 3962 consensus_queue.cc:237] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } }
I20260923 22:08:03.790649 3391 catalog_manager.cc:5495] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3e73b8c6373400abcfaf68ea9238ea1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 44941 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:03.800402 3688 raft_consensus.cc:1261] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:03.800433 3549 raft_consensus.cc:1261] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Refusing update from remote peer e3e73b8c6373400abcfaf68ea9238ea1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:03.800628 3962 consensus_queue.cc:1035] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd5cffbaec4467c817942dcdf60a248" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:03.800705 3995 consensus_queue.cc:1035] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb10a317ad34e5fb5c436b5b250e190" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:08:03.804705 3954 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 22:08:05.319366 3392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:05.319481 3392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:05.320293 3392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3141988640154b99af3c9f0f30e020d0: Sending DeleteTablet for 3 replicas of tablet 14fc7521bb04420e99d9b2df01e85efd
I20260923 22:08:05.320590 3669 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc7521bb04420e99d9b2df01e85efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:05 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:40514
I20260923 22:08:05.320612 3809 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc7521bb04420e99d9b2df01e85efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:05 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:34556
I20260923 22:08:05.320614 3529 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fc7521bb04420e99d9b2df01e85efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:05 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260923 22:08:05.320799 4027 tablet_replica.cc:330] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190: stopping tablet replica
I20260923 22:08:05.320874 4025 tablet_replica.cc:330] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: stopping tablet replica
I20260923 22:08:05.320884 4027 raft_consensus.cc:2229] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:05.320909 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 3464
I20260923 22:08:05.320936 4025 raft_consensus.cc:2229] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:05.320953 4027 raft_consensus.cc:2258] T 14fc7521bb04420e99d9b2df01e85efd P fcb10a317ad34e5fb5c436b5b250e190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:05.322320 4025 raft_consensus.cc:2258] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:05.322533 4025 ts_tablet_manager.cc:1905] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:05.324283 4025 ts_tablet_manager.cc:1918] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 22:08:05.324345 4025 log.cc:1196] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/14fc7521bb04420e99d9b2df01e85efd
I20260923 22:08:05.324555 4025 ts_tablet_manager.cc:1939] T 14fc7521bb04420e99d9b2df01e85efd P e3e73b8c6373400abcfaf68ea9238ea1: Deleting consensus metadata
I20260923 22:08:05.325018 3379 catalog_manager.cc:4841] TS e3e73b8c6373400abcfaf68ea9238ea1 (127.1.161.131:44941): tablet 14fc7521bb04420e99d9b2df01e85efd (table test-table [id=9e58a4884c2a4732a76ddd3922620c7f]) successfully deleted
I20260923 22:08:05.327296 4026 tablet_replica.cc:330] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: stopping tablet replica
I20260923 22:08:05.327382 4026 raft_consensus.cc:2229] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:05.327455 4026 raft_consensus.cc:2258] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 22:08:05.327678 3379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:42749 (error 108)
I20260923 22:08:05.327713 4026 ts_tablet_manager.cc:1905] T 14fc7521bb04420e99d9b2df01e85efd P cbd5cffbaec4467c817942dcdf60a248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:05.327891 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 3604
I20260923 22:08:05.329084 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:05.328485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:42749 (local address 127.0.0.1:39786)
0923 22:08:05.329017 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:05.329054 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:42749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":54,"threads_started":1}
W20260923 22:08:05.329226 3379 catalog_manager.cc:4584] TS fcb10a317ad34e5fb5c436b5b250e190 (127.1.161.129:42749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14fc7521bb04420e99d9b2df01e85efd: Network error: Client connection negotiation failed: client connection to 127.1.161.129:42749: connect: Connection refused (error 111)
I20260923 22:08:05.332324 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:05.332226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35881 (local address 127.0.0.1:40526)
0923 22:08:05.332272 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:05.332294 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.130:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:08:05.332335 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 3744
I20260923 22:08:05.338084 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 3360
2026-09-23T22:08:05Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5192 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:05 dist-test-slave-npvd krb5kdc[4032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:05 dist-test-slave-npvd krb5kdc[4032](info): set up 2 sockets
Sep 23 22:08:05 dist-test-slave-npvd krb5kdc[4032](info): commencing operation
krb5kdc: starting...
W20260923 22:08:06.266309 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.908s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:06.365552 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41497
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.435801 4045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.436025 4045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.436059 4045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.437144 4045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:06.437203 4045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.437227 4045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.437255 4045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:06.437283 4045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:06.438619 4045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:06.438869 4045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.439051 4045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.443390 4051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.443418 4050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.443532 4053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.443769 4045 server_base.cc:1034] running on GCE node
I20260923 22:08:06.443990 4045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.444248 4045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.445412 4045 hybrid_clock.cc:648] HybridClock initialized: now 1790201286445402 us; error 34 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.447623 4045 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:06.447985 4045 webserver.cc:460] Webserver started at http://127.1.161.190:41855/ using document root <none> and password file <none>
I20260923 22:08:06.448179 4045 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.448239 4045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.448369 4045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.449249 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "863fcd2e02ff48ddb4a9414ce57e69e4"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "2615651842d3d39949a0866745505b74"
server_key_iv: "46c1744628205b478dbd2428e8f9e772"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.449563 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "863fcd2e02ff48ddb4a9414ce57e69e4"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "2615651842d3d39949a0866745505b74"
server_key_iv: "46c1744628205b478dbd2428e8f9e772"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.450625 4045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:06.451332 4060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.451493 4045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.451596 4045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "863fcd2e02ff48ddb4a9414ce57e69e4"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "2615651842d3d39949a0866745505b74"
server_key_iv: "46c1744628205b478dbd2428e8f9e772"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.451675 4045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.461431 4045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.462925 4045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.463066 4045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.466213 4045 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41497
I20260923 22:08:06.466250 4112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41497 every 8 connection(s)
I20260923 22:08:06.466506 4045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:06.467370 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:06.469142 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4045
I20260923 22:08:06.469230 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:06.469285 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: Bootstrap starting.
I20260923 22:08:06.469377 1670 external_mini_cluster.cc:1428] Setting key 0c3f4f3268f9f9b3638aac4d6f7a715e
I20260923 22:08:06.469900 4113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:06.470170 4113 log.cc:824] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:06.471414 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: No bootstrap required, opened a new log
I20260923 22:08:06.473083 4113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } }
I20260923 22:08:06.473208 4113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:06.473246 4113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863fcd2e02ff48ddb4a9414ce57e69e4, State: Initialized, Role: FOLLOWER
I20260923 22:08:06.473371 4113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } }
I20260923 22:08:06.473443 4113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:06.473466 4113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:06.473496 4113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:06.474066 4113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } }
I20260923 22:08:06.474179 4113 leader_election.cc:302] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863fcd2e02ff48ddb4a9414ce57e69e4; no voters:
I20260923 22:08:06.474315 4113 leader_election.cc:288] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:06.474359 4118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:06.474502 4118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [term 1 LEADER]: Becoming Leader. State: Replica: 863fcd2e02ff48ddb4a9414ce57e69e4, State: Running, Role: LEADER
I20260923 22:08:06.474557 4113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:06.475224 4118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } }
I20260923 22:08:06.475737 4120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } } }
I20260923 22:08:06.475831 4120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [sys.catalog]: This master's current role is: LEADER
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:06.476091 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863fcd2e02ff48ddb4a9414ce57e69e4. Latest consensus state: current_term: 1 leader_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863fcd2e02ff48ddb4a9414ce57e69e4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41497 } } }
I20260923 22:08:06.476326 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4 [sys.catalog]: This master's current role is: LEADER
W20260923 22:08:06.476358 4132 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:06.476392 4132 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:06.476483 4133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:06.478703 4133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:06.479810 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.469789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.1.161.190:41497)
0923 22:08:06.469933 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.469955 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.469982 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:06.470386 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.470392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.470431 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.470542 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.471152 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.472532 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.473318 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.473829 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.473945 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.476666 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.476669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.476670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.476680 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.478301 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.478608 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.478611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.478612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.478707 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.478933 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.478934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.478934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.479228 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.479341 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.479461 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.479599 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260923 22:08:06.480141 4133 catalog_manager.cc:1347] Generated new cluster ID: 90eae9e7dd8246bfb1691013bc314103
I20260923 22:08:06.480192 4133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:06.494105 4133 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:06.494645 4133 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:06.501108 4133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863fcd2e02ff48ddb4a9414ce57e69e4: Generated new TSK 0
I20260923 22:08:06.501259 4133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:06.531987 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.605208 4139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.605376 4139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.605396 4139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.606365 4139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.606395 4139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.606418 4139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:06.607599 4139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:06.607754 4139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.607887 4139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:06.612247 4139 server_base.cc:1034] running on GCE node
W20260923 22:08:06.612233 4144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.612231 4147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.612205 4145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.612658 4139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.612823 4139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.613981 4139 hybrid_clock.cc:648] HybridClock initialized: now 1790201286613966 us; error 30 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.616060 4139 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:06.616403 4139 webserver.cc:460] Webserver started at http://127.1.161.129:39421/ using document root <none> and password file <none>
I20260923 22:08:06.616600 4139 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.616679 4139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.616812 4139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.617684 4139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "3744cf73f73943dc939f0004ef9ee677"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "1f5b37137965558e6ae3a1d728559fb8"
server_key_iv: "c1665c02e867771d27bb03080fdf1973"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.617998 4139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "3744cf73f73943dc939f0004ef9ee677"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "1f5b37137965558e6ae3a1d728559fb8"
server_key_iv: "c1665c02e867771d27bb03080fdf1973"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.619105 4139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:06.619771 4154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.619926 4139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.620008 4139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "3744cf73f73943dc939f0004ef9ee677"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "1f5b37137965558e6ae3a1d728559fb8"
server_key_iv: "c1665c02e867771d27bb03080fdf1973"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.620078 4139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.630716 4139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.632702 4139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.632836 4139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.633020 4139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.633270 4139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.633317 4139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.633347 4139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.633368 4139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.638314 4139 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:35215
I20260923 22:08:06.638373 4267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:35215 every 8 connection(s)
I20260923 22:08:06.638633 4139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:06.645794 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4139
I20260923 22:08:06.645886 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:06.646052 1670 external_mini_cluster.cc:1428] Setting key 35711d39534f7fa440c98bfd027fb592
I20260923 22:08:06.646056 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.639245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:33289 (local address 127.1.161.190:41497)
0923 22:08:06.639279 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.639281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.639434 (+ 153us) server_negotiation.cc:407] Connection header received
0923 22:08:06.639804 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.639805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.639814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.639858 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.640547 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.641010 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.641888 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.642090 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.642149 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.643805 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.643806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.643807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.643812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.645001 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.645297 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.645297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.645298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.645339 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.645511 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.645511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.645511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.645604 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.645646 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.645929 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.646002 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:08:06.646226 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.639226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41497 (local address 127.1.161.129:33289)
0923 22:08:06.639350 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.639377 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.639734 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.639897 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.639900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.640336 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.640480 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.640483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.641086 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.641087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.641818 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.641820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.642166 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.642167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.642260 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.642596 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.642603 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.643733 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.645064 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.645068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.645069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.645243 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.645387 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.645388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.645388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.645463 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.645662 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.645667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.645799 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.645899 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.646031 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
I20260923 22:08:06.646658 4268 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41497
I20260923 22:08:06.646736 4268 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.646876 4268 heartbeater.cc:511] Master 127.1.161.190:41497 requested a full tablet report, sending...
I20260923 22:08:06.647406 4076 ts_manager.cc:194] Registered new tserver with Master: 3744cf73f73943dc939f0004ef9ee677 (127.1.161.129:35215)
I20260923 22:08:06.648057 4076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:33289
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:06.669386 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.740509 4273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.740677 4273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.740697 4273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.741767 4273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.741801 4273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.741824 4273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:06.743091 4273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:06.743294 4273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.743451 4273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.747560 4281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.747572 4278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.747589 4273 server_base.cc:1034] running on GCE node
W20260923 22:08:06.747563 4279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.747937 4273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.748162 4273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.749289 4273 hybrid_clock.cc:648] HybridClock initialized: now 1790201286749271 us; error 34 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.751432 4273 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:06.751827 4273 webserver.cc:460] Webserver started at http://127.1.161.130:45225/ using document root <none> and password file <none>
I20260923 22:08:06.752043 4273 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.752120 4273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.752271 4273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.753232 4273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "4a900b6b19bb455aa4238ad8acc17b07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "b55864c907f0ef539ad80e72a80b6bc6"
server_key_iv: "957247cf65601aa416b01d017f0eeaf0"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.753538 4273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "4a900b6b19bb455aa4238ad8acc17b07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "b55864c907f0ef539ad80e72a80b6bc6"
server_key_iv: "957247cf65601aa416b01d017f0eeaf0"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.754709 4273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:06.755359 4288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.755501 4273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.755560 4273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "4a900b6b19bb455aa4238ad8acc17b07"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "b55864c907f0ef539ad80e72a80b6bc6"
server_key_iv: "957247cf65601aa416b01d017f0eeaf0"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.755604 4273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.762391 4273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.763864 4273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.763948 4273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.764076 4273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.764310 4273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.764339 4273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.764355 4273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.764400 4273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.769174 4273 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:37197
I20260923 22:08:06.769212 4401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:37197 every 8 connection(s)
I20260923 22:08:06.769480 4273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:06.773007 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4273
I20260923 22:08:06.773088 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:06.773268 1670 external_mini_cluster.cc:1428] Setting key 9f724ee32ddac579b0f22458822141ec
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:06.777957 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.770129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:35353 (local address 127.1.161.190:41497)
0923 22:08:06.770183 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.770185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.770376 (+ 191us) server_negotiation.cc:407] Connection header received
0923 22:08:06.770717 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.770718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.770726 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.770770 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.771606 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.772117 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.772956 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.773171 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.773224 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.775239 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.775240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.775240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.775246 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.776685 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.777071 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.777072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.777072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.777133 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.777380 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.777381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.777381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.777482 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.777541 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.777774 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.777849 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:08:06.778087 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.770107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41497 (local address 127.1.161.130:35353)
0923 22:08:06.770262 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.770297 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.770650 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.770808 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.770810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.771373 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.771534 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.771538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.772188 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.772189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.772878 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.772881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.773236 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.773237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.773366 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.773822 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.773836 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.775129 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.776766 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.776772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.776774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.776985 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.777198 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.777199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.777199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.777322 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.777543 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.777549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.777661 (+ 112us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.777745 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.777883 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260923 22:08:06.778442 4402 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41497
I20260923 22:08:06.778513 4402 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.778652 4402 heartbeater.cc:511] Master 127.1.161.190:41497 requested a full tablet report, sending...
I20260923 22:08:06.778980 4076 ts_manager.cc:194] Registered new tserver with Master: 4a900b6b19bb455aa4238ad8acc17b07 (127.1.161.130:37197)
I20260923 22:08:06.779458 4076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:35353
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:06.796167 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:06.865293 4407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:06.865442 4407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:06.865459 4407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:06.866353 4407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:06.866382 4407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:06.866403 4407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:06.867530 4407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:06.867682 4407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:06.867811 4407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:06.871866 4412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.871973 4407 server_base.cc:1034] running on GCE node
W20260923 22:08:06.871873 4415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:06.871873 4413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:06.872273 4407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:06.872511 4407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:06.873642 4407 hybrid_clock.cc:648] HybridClock initialized: now 1790201286873632 us; error 23 us; skew 500 ppm
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:06.875901 4407 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:06.876243 4407 webserver.cc:460] Webserver started at http://127.1.161.131:40869/ using document root <none> and password file <none>
I20260923 22:08:06.876475 4407 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:06.876516 4407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:06.876639 4407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:06.877439 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "88a3121219b04dbcaf1563bea6f8d314"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "8acdad5ba1c7baa441886ede2d2df49f"
server_key_iv: "355c101e7a9f69db06d250ba48bae4fb"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.877732 4407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "88a3121219b04dbcaf1563bea6f8d314"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "8acdad5ba1c7baa441886ede2d2df49f"
server_key_iv: "355c101e7a9f69db06d250ba48bae4fb"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.878790 4407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:06.879462 4422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.879617 4407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:06.879678 4407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "88a3121219b04dbcaf1563bea6f8d314"
format_stamp: "Formatted at 2026-09-23 22:08:06 on dist-test-slave-npvd"
server_key: "8acdad5ba1c7baa441886ede2d2df49f"
server_key_iv: "355c101e7a9f69db06d250ba48bae4fb"
server_key_version: "encryptionkey@0"
I20260923 22:08:06.879750 4407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:06.892212 4407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:06.893607 4407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:06.893723 4407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:06.893908 4407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:06.894145 4407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:06.894172 4407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.894188 4407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:06.894227 4407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:06.898977 4407 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:36277
I20260923 22:08:06.899006 4535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:36277 every 8 connection(s)
I20260923 22:08:06.899293 4407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:06.899501 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4407
I20260923 22:08:06.899610 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:06.899788 1670 external_mini_cluster.cc:1428] Setting key a0e787718bed908e6ba244f40707deb5
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201286, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:06.906520 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.899940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60931 (local address 127.1.161.190:41497)
0923 22:08:06.899997 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.899999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.900187 (+ 188us) server_negotiation.cc:407] Connection header received
0923 22:08:06.900651 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.900652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.900659 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.900698 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.901439 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.901904 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.902715 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.902864 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.902907 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.904551 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:06.904552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:06.904552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:06.904557 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:06.905624 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.905907 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.905908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.905908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.905949 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:06.906114 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:06.906115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:06.906115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:06.906200 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:06.906240 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:06.906421 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 22:08:06.906468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:06.906612 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:06.899938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41497 (local address 127.1.161.131:60931)
0923 22:08:06.900074 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:06.900110 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:06.900568 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:06.900738 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:06.900740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:06.901215 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:06.901372 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.901375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.901977 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.901978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.902656 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:06.902658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:06.902922 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:06.902923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:06.902999 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:06.903401 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:06.903408 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:06.904489 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:06.905682 (+ 1193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.905686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.905688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.905852 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.905997 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:06.905998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:06.905998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:06.906069 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:06.906246 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:06.906249 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:06.906328 (+ 79us) client_negotiation.cc:769] Sending connection context
0923 22:08:06.906391 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:06.906478 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260923 22:08:06.907078 4536 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41497
I20260923 22:08:06.907153 4536 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:06.907334 4536 heartbeater.cc:511] Master 127.1.161.190:41497 requested a full tablet report, sending...
I20260923 22:08:06.907639 4076 ts_manager.cc:194] Registered new tserver with Master: 88a3121219b04dbcaf1563bea6f8d314 (127.1.161.131:36277)
I20260923 22:08:06.907909 4076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:60931
I20260923 22:08:06.911326 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:08:06.920243 4116 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:06.919563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.1.161.190:41497)
0923 22:08:06.919626 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:06.919628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:06.919712 (+ 84us) server_negotiation.cc:407] Connection header received
0923 22:08:06.919773 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:06.919774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:06.919782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:06.919827 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:06.920201 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53346: BlockingRecv error: recv got EOF from 127.0.0.1:53346 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:06.921172 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4139
I20260923 22:08:06.925238 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4273
I20260923 22:08:06.928604 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4407
I20260923 22:08:06.932240 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4045
2026-09-23T22:08:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1592 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4548](info): set up 2 sockets
Sep 23 22:08:06 dist-test-slave-npvd krb5kdc[4548](info): commencing operation
krb5kdc: starting...
W20260923 22:08:07.862663 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.912s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:07 dist-test-slave-npvd krb5kdc[4548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:07.961632 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:34707
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:34707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:08.035069 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:08.035332 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:08.035383 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:08.036546 4561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:08.036592 4561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:08.036609 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:08.036712 4561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:08.036778 4561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:08.038333 4561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:34707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:34707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:08.038552 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:08.038785 4561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:08.043459 4566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:08.043473 4567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:08.043473 4569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:08.043671 4561 server_base.cc:1034] running on GCE node
I20260923 22:08:08.043889 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:08.044099 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:08.045240 4561 hybrid_clock.cc:648] HybridClock initialized: now 1790201288045225 us; error 32 us; skew 500 ppm
Sep 23 22:08:08 dist-test-slave-npvd krb5kdc[4548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201288, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:08.047500 4561 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:08.047889 4561 webserver.cc:460] Webserver started at http://127.1.161.190:43379/ using document root <none> and password file <none>
I20260923 22:08:08.048091 4561 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:08.048152 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:08.048282 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:08.049139 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "6e1ca2620d6d479d956d9b40be6ba24c"
format_stamp: "Formatted at 2026-09-23 22:08:08 on dist-test-slave-npvd"
server_key: "d45bdb83f3bbec151e7283fbccb2ccd4"
server_key_iv: "8f62421790524a58d53bae734a85b46a"
server_key_version: "encryptionkey@0"
I20260923 22:08:08.049448 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "6e1ca2620d6d479d956d9b40be6ba24c"
format_stamp: "Formatted at 2026-09-23 22:08:08 on dist-test-slave-npvd"
server_key: "d45bdb83f3bbec151e7283fbccb2ccd4"
server_key_iv: "8f62421790524a58d53bae734a85b46a"
server_key_version: "encryptionkey@0"
I20260923 22:08:08.050606 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:08.051354 4576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:08.051533 4561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:08.051630 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "6e1ca2620d6d479d956d9b40be6ba24c"
format_stamp: "Formatted at 2026-09-23 22:08:08 on dist-test-slave-npvd"
server_key: "d45bdb83f3bbec151e7283fbccb2ccd4"
server_key_iv: "8f62421790524a58d53bae734a85b46a"
server_key_version: "encryptionkey@0"
I20260923 22:08:08.051714 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:08.070372 4561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:08.072140 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:08.072293 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:08.075924 4561 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:34707
I20260923 22:08:08.075956 4628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:34707 every 8 connection(s)
I20260923 22:08:08.076262 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:08.077028 4629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:08.078963 4629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c: Bootstrap starting.
I20260923 22:08:08.079525 4629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:08.079816 4629 log.cc:824] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:08.080417 4629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c: No bootstrap required, opened a new log
I20260923 22:08:08.081851 4629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } }
I20260923 22:08:08.081985 4629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:08.082007 4629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1ca2620d6d479d956d9b40be6ba24c, State: Initialized, Role: FOLLOWER
I20260923 22:08:08.082144 4629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } }
I20260923 22:08:08.082221 4629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:08.082244 4629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:08.082312 4629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:08.082994 4629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } }
I20260923 22:08:08.083086 4629 leader_election.cc:302] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e1ca2620d6d479d956d9b40be6ba24c; no voters:
I20260923 22:08:08.083295 4629 leader_election.cc:288] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:08.083338 4632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:08.083535 4632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [term 1 LEADER]: Becoming Leader. State: Replica: 6e1ca2620d6d479d956d9b40be6ba24c, State: Running, Role: LEADER
I20260923 22:08:08.083539 4629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:08.083635 4632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } }
I20260923 22:08:08.084038 4633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } } }
I20260923 22:08:08.084044 4634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e1ca2620d6d479d956d9b40be6ba24c. Latest consensus state: current_term: 1 leader_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1ca2620d6d479d956d9b40be6ba24c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34707 } } }
I20260923 22:08:08.084167 4633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:08.084173 4634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:08.084453 4641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:08.085003 4641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:08.085438 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4561
I20260923 22:08:08.085551 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:08.085732 1670 external_mini_cluster.cc:1428] Setting key fe71f1a9d991c63f3458a9d1e698e6fe
I20260923 22:08:08.087519 4641 catalog_manager.cc:1347] Generated new cluster ID: 950928f343bf4b78a4baa3fc2937aa79
I20260923 22:08:08.087577 4641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:08 dist-test-slave-npvd krb5kdc[4548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:08.097709 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:08.086118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.1.161.190:34707)
0923 22:08:08.086309 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:08.086317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:08.086346 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:08:08.086566 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:08.086573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:08.086630 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:08.086750 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:08.087543 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:08.089005 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:08.090029 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:08.090664 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:08.090765 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:08.093823 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:08.093827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:08.093827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:08.093839 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:08.095771 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:08.096192 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:08.096195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:08.096196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:08.096293 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:08.096486 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:08.096487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:08.096487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:08.096767 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:08.096923 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:08.097379 (+ 456us) server_negotiation.cc:299] Negotiation successful
0923 22:08:08.097497 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
I20260923 22:08:08.098966 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:08.100200 4641 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:08.100838 4641 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:08.104612 4641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e1ca2620d6d479d956d9b40be6ba24c: Generated new TSK 0
I20260923 22:08:08.104758 4641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:08.113643 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:08.110249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42656 (local address 127.1.161.190:34707)
0923 22:08:08.110283 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:08.110285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:08.110413 (+ 128us) server_negotiation.cc:407] Connection header received
0923 22:08:08.110493 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:08.110494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:08.110504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:08.110552 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:08.110901 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:08.111461 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:08.112212 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:08.112425 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:08.112499 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:08.113207 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:08.113208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:08.113208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:08.113395 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:08.113408 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:08.113414 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:08.113478 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:08.113540 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 22:08:08.113561 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:08.116101 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4561
2026-09-23T22:08:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1182 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:08 dist-test-slave-npvd krb5kdc[4662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:08 dist-test-slave-npvd krb5kdc[4662](info): set up 2 sockets
Sep 23 22:08:08 dist-test-slave-npvd krb5kdc[4662](info): commencing operation
krb5kdc: starting...
W20260923 22:08:09.050480 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:09.149497 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33507
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:09.218959 4675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:09.219116 4675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:09.219135 4675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:09.220211 4675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:09.220250 4675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:09.220265 4675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:09.220278 4675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:09.220330 4675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:09.221676 4675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:09.221899 4675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:09.222119 4675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:09.226298 4680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.226362 4675 server_base.cc:1034] running on GCE node
W20260923 22:08:09.226307 4681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:09.226311 4683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.226682 4675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:09.226924 4675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:09.228063 4675 hybrid_clock.cc:648] HybridClock initialized: now 1790201289228056 us; error 24 us; skew 500 ppm
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:09.230088 4675 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:09.230464 4675 webserver.cc:460] Webserver started at http://127.1.161.190:41373/ using document root <none> and password file <none>
I20260923 22:08:09.230661 4675 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:09.230719 4675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:09.230831 4675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:09.231755 4675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "b584197bb9cc4c6eacbc18bde0214148"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "8a8545596e0463281c38b610c30b7143"
server_key_iv: "d58a8d88f92eca37d4e137c2a5c9cbc3"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.232090 4675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "b584197bb9cc4c6eacbc18bde0214148"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "8a8545596e0463281c38b610c30b7143"
server_key_iv: "d58a8d88f92eca37d4e137c2a5c9cbc3"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.233337 4675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:09.233933 4690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.234072 4675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:09.234156 4675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "b584197bb9cc4c6eacbc18bde0214148"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "8a8545596e0463281c38b610c30b7143"
server_key_iv: "d58a8d88f92eca37d4e137c2a5c9cbc3"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.234228 4675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:09.245882 4675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:09.247666 4675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:09.247797 4675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:09.251112 4675 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:33507
I20260923 22:08:09.251138 4742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:33507 every 8 connection(s)
I20260923 22:08:09.251449 4675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:09.252323 4743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:09.253054 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4675
I20260923 22:08:09.253158 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:09.253316 1670 external_mini_cluster.cc:1428] Setting key a0af6f73442e490236129c3ae9215b69
I20260923 22:08:09.255059 4743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148: Bootstrap starting.
I20260923 22:08:09.255578 4743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:09.255836 4743 log.cc:824] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:09.256405 4743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148: No bootstrap required, opened a new log
I20260923 22:08:09.258002 4743 raft_consensus.cc:348] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } }
I20260923 22:08:09.258160 4743 raft_consensus.cc:374] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:09.258196 4743 raft_consensus.cc:729] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b584197bb9cc4c6eacbc18bde0214148, State: Initialized, Role: FOLLOWER
I20260923 22:08:09.258299 4743 consensus_queue.cc:260] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } }
I20260923 22:08:09.258380 4743 raft_consensus.cc:388] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:09.258409 4743 raft_consensus.cc:482] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:09.258456 4743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:09.259039 4743 raft_consensus.cc:504] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } }
I20260923 22:08:09.259160 4743 leader_election.cc:302] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b584197bb9cc4c6eacbc18bde0214148; no voters:
I20260923 22:08:09.259326 4743 leader_election.cc:288] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:09.259372 4748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:09.259501 4748 raft_consensus.cc:686] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [term 1 LEADER]: Becoming Leader. State: Replica: b584197bb9cc4c6eacbc18bde0214148, State: Running, Role: LEADER
I20260923 22:08:09.259553 4743 sys_catalog.cc:564] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:09.259562 4748 consensus_queue.cc:237] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } }
I20260923 22:08:09.259853 4748 sys_catalog.cc:455] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b584197bb9cc4c6eacbc18bde0214148. Latest consensus state: current_term: 1 leader_uuid: "b584197bb9cc4c6eacbc18bde0214148" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } } }
I20260923 22:08:09.259913 4749 sys_catalog.cc:455] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b584197bb9cc4c6eacbc18bde0214148" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b584197bb9cc4c6eacbc18bde0214148" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33507 } } }
I20260923 22:08:09.259923 4748 sys_catalog.cc:458] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:09.259984 4749 sys_catalog.cc:458] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:09.260221 4756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:09.260643 4756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:09.262769 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.253750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.1.161.190:33507)
0923 22:08:09.253890 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.253897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.253928 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:09.254268 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.254273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.254310 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.254400 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:09.255052 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.255876 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.256547 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.256889 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.256993 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.259096 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:09.259099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:09.259099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:09.259123 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:09.261307 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.261728 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.261729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.261730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.261802 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.262000 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.262001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.262001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.262194 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:09.262298 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.262444 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.262534 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260923 22:08:09.264072 4756 catalog_manager.cc:1347] Generated new cluster ID: fd4435bc5bb34e2b83d3dca95674cb53
I20260923 22:08:09.264123 4756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:09.283581 4756 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:09.283999 4756 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:09.286270 4756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b584197bb9cc4c6eacbc18bde0214148: Generated new TSK 0
I20260923 22:08:09.286408 4756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:09.311774 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:09.385771 4769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:09.385974 4769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:09.386009 4769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:09.387127 4769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:09.387215 4769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:09.387274 4769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:09.388551 4769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:09.388770 4769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:09.388958 4769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:09.393321 4777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:09.393337 4775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.393340 4769 server_base.cc:1034] running on GCE node
W20260923 22:08:09.393299 4774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.393662 4769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:09.393826 4769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:09.394975 4769 hybrid_clock.cc:648] HybridClock initialized: now 1790201289394955 us; error 30 us; skew 500 ppm
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:09.397368 4769 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:09.397675 4769 webserver.cc:460] Webserver started at http://127.1.161.129:33157/ using document root <none> and password file <none>
I20260923 22:08:09.397814 4769 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:09.397846 4769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:09.397917 4769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:09.398692 4769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "d9ca3c1aecb645ccaad4867f494b2357"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "e61e253a6a2802d0eda556fb8646e19b"
server_key_iv: "88b85fca5b53c6cd2c81c6616f6f2552"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.398931 4769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "d9ca3c1aecb645ccaad4867f494b2357"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "e61e253a6a2802d0eda556fb8646e19b"
server_key_iv: "88b85fca5b53c6cd2c81c6616f6f2552"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.400125 4769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:09.400759 4784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.400883 4769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.400957 4769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d9ca3c1aecb645ccaad4867f494b2357"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "e61e253a6a2802d0eda556fb8646e19b"
server_key_iv: "88b85fca5b53c6cd2c81c6616f6f2552"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.401002 4769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:09.420264 4769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:09.421981 4769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:09.422083 4769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:09.422258 4769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:09.422549 4769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:09.422580 4769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.422600 4769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:09.422646 4769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.428217 4769 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:35551
I20260923 22:08:09.428290 4897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:35551 every 8 connection(s)
I20260923 22:08:09.428571 4769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:09.435868 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4769
I20260923 22:08:09.435971 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:09.436182 1670 external_mini_cluster.cc:1428] Setting key cc340f10400228fac78f7cd1ac6ccbb1
I20260923 22:08:09.437306 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.429196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:33199 (local address 127.1.161.190:33507)
0923 22:08:09.429238 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.429240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.429424 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:08:09.429886 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.429887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.429895 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.429951 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:09.430802 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.431349 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.432280 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.432475 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.432555 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.434568 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:09.434569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:09.434570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:09.434576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:09.435836 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.436296 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.436297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.436297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.436359 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.436621 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.436622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.436622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.436750 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:09.436813 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.437102 (+ 289us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.437186 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:09.437377 4900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.429155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33507 (local address 127.1.161.129:33199)
0923 22:08:09.429301 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.429340 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:09.429793 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:09.430000 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:09.430003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:09.430511 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:09.430721 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.430728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.431416 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.431417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.432206 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.432208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.432566 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.432567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.432679 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.433174 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:09.433182 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:09.434463 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:09.435907 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.435913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.435915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.436216 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.436436 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.436437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.436437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.436568 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.436822 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:09.436827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:09.436965 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 22:08:09.437045 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:08:09.437157 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260923 22:08:09.437912 4898 heartbeater.cc:347] Connected to a master server at 127.1.161.190:33507
I20260923 22:08:09.438021 4898 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:09.438205 4898 heartbeater.cc:511] Master 127.1.161.190:33507 requested a full tablet report, sending...
I20260923 22:08:09.438683 4707 ts_manager.cc:194] Registered new tserver with Master: d9ca3c1aecb645ccaad4867f494b2357 (127.1.161.129:35551)
I20260923 22:08:09.439328 4707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:33199
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:09.457943 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:09.528730 4903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:09.528906 4903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:09.528939 4903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:09.529891 4903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:09.529949 4903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:09.529987 4903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:09.531260 4903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:09.531467 4903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:09.531649 4903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:09.536172 4909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.536278 4903 server_base.cc:1034] running on GCE node
W20260923 22:08:09.536213 4911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:09.536357 4908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.536585 4903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:09.536806 4903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:09.537954 4903 hybrid_clock.cc:648] HybridClock initialized: now 1790201289537951 us; error 32 us; skew 500 ppm
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:09.540355 4903 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:09.540747 4903 webserver.cc:460] Webserver started at http://127.1.161.130:38499/ using document root <none> and password file <none>
I20260923 22:08:09.541002 4903 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:09.541059 4903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:09.541173 4903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:09.542099 4903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "050f7413babf4316bc4b31162de186ac"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "923f4413f7d98d6732374bd607e9b57e"
server_key_iv: "1a651aff8e29128696beb792b29e4e62"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.542444 4903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "050f7413babf4316bc4b31162de186ac"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "923f4413f7d98d6732374bd607e9b57e"
server_key_iv: "1a651aff8e29128696beb792b29e4e62"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.543875 4903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:09.544514 4918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.544658 4903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.544740 4903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "050f7413babf4316bc4b31162de186ac"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "923f4413f7d98d6732374bd607e9b57e"
server_key_iv: "1a651aff8e29128696beb792b29e4e62"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.544837 4903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:09.554304 4903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:09.555972 4903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:09.556100 4903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:09.556272 4903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:09.556509 4903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:09.556555 4903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.556582 4903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:09.556605 4903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.561936 4903 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:37805
I20260923 22:08:09.562004 5031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:37805 every 8 connection(s)
I20260923 22:08:09.562289 4903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:09.570014 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.562956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:47577 (local address 127.1.161.190:33507)
0923 22:08:09.563016 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.563018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.563217 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:08:09.563600 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.563601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.563610 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.563676 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:09.564473 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.564929 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.565746 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.565903 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.565946 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.567799 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:09.567800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:09.567800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:09.567805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:09.568936 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.569272 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.569273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.569273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.569319 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.569530 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.569530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.569531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.569637 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:09.569690 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.569900 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.569954 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:09.570147 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.562916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33507 (local address 127.1.161.130:47577)
0923 22:08:09.563059 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.563092 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:09.563527 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:09.563727 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:09.563730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:09.564245 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:09.564403 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.564407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.564998 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.564999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.565677 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.565679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.565965 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.565966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.566051 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.566479 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:09.566487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:09.567725 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:09.569006 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.569011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.569012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.569206 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.569376 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.569376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.569377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.569464 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.569690 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:09.569694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:09.569798 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:08:09.569873 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:08:09.569976 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260923 22:08:09.570616 5032 heartbeater.cc:347] Connected to a master server at 127.1.161.190:33507
I20260923 22:08:09.570695 5032 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:09.570815 5032 heartbeater.cc:511] Master 127.1.161.190:33507 requested a full tablet report, sending...
I20260923 22:08:09.571113 4707 ts_manager.cc:194] Registered new tserver with Master: 050f7413babf4316bc4b31162de186ac (127.1.161.130:37805)
I20260923 22:08:09.571491 4707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:47577
I20260923 22:08:09.571800 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 4903
I20260923 22:08:09.571907 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:09.572082 1670 external_mini_cluster.cc:1428] Setting key b8156e39ddf3a74d181d61fc2dc39f54
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:09.594017 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:09.667064 5037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:09.667277 5037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:09.667315 5037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:09.668414 5037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:09.668479 5037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:09.668524 5037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:09.670228 5037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:09.670495 5037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:09.670702 5037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:09.675453 5043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.675577 5037 server_base.cc:1034] running on GCE node
W20260923 22:08:09.675503 5045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:09.675657 5042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:09.675873 5037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:09.676137 5037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:09.677292 5037 hybrid_clock.cc:648] HybridClock initialized: now 1790201289677280 us; error 25 us; skew 500 ppm
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:09.679559 5037 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:09.679953 5037 webserver.cc:460] Webserver started at http://127.1.161.131:33741/ using document root <none> and password file <none>
I20260923 22:08:09.680207 5037 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:09.680271 5037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:09.680425 5037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:09.681300 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "1ee715ad89064abebcab1913e6c040c1"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "4998208e4699347d3b7e92e24b637927"
server_key_iv: "db7880ee863625cf5334cad30efbdb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.681612 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "1ee715ad89064abebcab1913e6c040c1"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "4998208e4699347d3b7e92e24b637927"
server_key_iv: "db7880ee863625cf5334cad30efbdb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.682822 5037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:09.683516 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.683666 5037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.683751 5037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "1ee715ad89064abebcab1913e6c040c1"
format_stamp: "Formatted at 2026-09-23 22:08:09 on dist-test-slave-npvd"
server_key: "4998208e4699347d3b7e92e24b637927"
server_key_iv: "db7880ee863625cf5334cad30efbdb5a"
server_key_version: "encryptionkey@0"
I20260923 22:08:09.683826 5037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:09.699755 5037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:09.701375 5037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:09.701552 5037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:09.701756 5037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:09.702056 5037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:09.702108 5037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.702140 5037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:09.702168 5037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:09.707549 5037 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:44347
I20260923 22:08:09.707592 5165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:44347 every 8 connection(s)
I20260923 22:08:09.707839 5037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:09.716395 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.708442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:44111 (local address 127.1.161.190:33507)
0923 22:08:09.708526 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.708528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.708686 (+ 158us) server_negotiation.cc:407] Connection header received
0923 22:08:09.709198 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.709199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.709212 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.709262 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:09.710121 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.710605 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.711628 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.711861 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.711939 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.713819 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:09.713821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:09.713821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:09.713828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:09.714880 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.715314 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.715316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.715316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.715373 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.715613 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.715614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.715614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.715782 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:09.715845 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.716181 (+ 336us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.716272 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 22:08:09.716504 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.708373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33507 (local address 127.1.161.131:44111)
0923 22:08:09.708552 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:09.708588 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:09.709093 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:09.709334 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:09.709337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:09.709860 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:09.710025 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.710029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.710707 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.710709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.711544 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:09.711547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.711943 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.711944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:09.712060 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.712469 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:09.712477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:09.713721 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:09.714957 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.714962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.714964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.715218 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.715438 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:09.715438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:09.715439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:09.715551 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:09.715836 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:09.715841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:09.716003 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 22:08:09.716131 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 22:08:09.716291 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260923 22:08:09.717175 5166 heartbeater.cc:347] Connected to a master server at 127.1.161.190:33507
I20260923 22:08:09.717288 5166 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:09.717463 5166 heartbeater.cc:511] Master 127.1.161.190:33507 requested a full tablet report, sending...
I20260923 22:08:09.717823 4707 ts_manager.cc:194] Registered new tserver with Master: 1ee715ad89064abebcab1913e6c040c1 (127.1.161.131:44347)
I20260923 22:08:09.717978 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5037
I20260923 22:08:09.718093 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:09.718276 1670 external_mini_cluster.cc:1428] Setting key 63b20aa46cb31e571154b8c86149530d
I20260923 22:08:09.718274 4707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:44111
I20260923 22:08:09.718993 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[4662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:09.740059 4746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:09.733384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56086 (local address 127.1.161.190:33507)
0923 22:08:09.733452 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:09.733454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:09.733464 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:09.733519 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:09.733520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:09.733530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:09.733581 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:09.734150 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.734606 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.735380 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:09.735570 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:09.735652 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:09.737758 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:09.737759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:09.737760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:09.737767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:09.738896 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.739267 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.739268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.739269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.739327 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:09.739526 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:09.739526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:09.739527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:09.739665 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:09.739784 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:09.739883 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:08:09.739961 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 22:08:09.740298 4707 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56086
I20260923 22:08:09.740725 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4769
I20260923 22:08:09.745132 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4903
I20260923 22:08:09.749277 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5037
I20260923 22:08:09.753487 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 4675
2026-09-23T22:08:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1639 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[5178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[5178](info): set up 2 sockets
Sep 23 22:08:09 dist-test-slave-npvd krb5kdc[5178](info): commencing operation
krb5kdc: starting...
W20260923 22:08:10.662518 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.890s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:10 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:10.761837 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41655
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:10.830715 5191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:10.830868 5191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:10.830885 5191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:10.831856 5191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:10.831890 5191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:10.831903 5191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:10.831939 5191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:10.831965 5191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:10.833441 5191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:10.833639 5191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:10.833814 5191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:10.837702 5197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:10.837781 5191 server_base.cc:1034] running on GCE node
W20260923 22:08:10.837699 5199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:10.837944 5196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:10.838191 5191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:10.838434 5191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:10.839558 5191 hybrid_clock.cc:648] HybridClock initialized: now 1790201290839557 us; error 35 us; skew 500 ppm
Sep 23 22:08:10 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201290, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:10.852861 5191 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:10.853261 5191 webserver.cc:460] Webserver started at http://127.1.161.190:43461/ using document root <none> and password file <none>
I20260923 22:08:10.853485 5191 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:10.853549 5191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:10.853689 5191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:10.854576 5191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "33f6308ac81a45b1be74a6a080b71924"
format_stamp: "Formatted at 2026-09-23 22:08:10 on dist-test-slave-npvd"
server_key: "afadc770cf3093881ddbcaaf3a7d042b"
server_key_iv: "cafb3da65ca355ee1b6d789a3fdf9bda"
server_key_version: "encryptionkey@0"
I20260923 22:08:10.854878 5191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "33f6308ac81a45b1be74a6a080b71924"
format_stamp: "Formatted at 2026-09-23 22:08:10 on dist-test-slave-npvd"
server_key: "afadc770cf3093881ddbcaaf3a7d042b"
server_key_iv: "cafb3da65ca355ee1b6d789a3fdf9bda"
server_key_version: "encryptionkey@0"
I20260923 22:08:10.856146 5191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:10.856927 5206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:10.857084 5191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:10.857164 5191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "33f6308ac81a45b1be74a6a080b71924"
format_stamp: "Formatted at 2026-09-23 22:08:10 on dist-test-slave-npvd"
server_key: "afadc770cf3093881ddbcaaf3a7d042b"
server_key_iv: "cafb3da65ca355ee1b6d789a3fdf9bda"
server_key_version: "encryptionkey@0"
I20260923 22:08:10.857211 5191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:10.907513 5191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:10.909601 5191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:10.909701 5191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:10.913571 5191 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41655
I20260923 22:08:10.913609 5258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41655 every 8 connection(s)
I20260923 22:08:10.914052 5191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:10.914616 5259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:10.915890 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5191
I20260923 22:08:10.915993 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:10.916139 1670 external_mini_cluster.cc:1428] Setting key 8587ed5ae51ab9a237f1e08510572e01
I20260923 22:08:10.916656 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924: Bootstrap starting.
I20260923 22:08:10.917160 5259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:10.917474 5259 log.cc:824] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:10.918081 5259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924: No bootstrap required, opened a new log
I20260923 22:08:10.919916 5259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } }
I20260923 22:08:10.920044 5259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:10.920081 5259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f6308ac81a45b1be74a6a080b71924, State: Initialized, Role: FOLLOWER
I20260923 22:08:10.920220 5259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } }
I20260923 22:08:10.920290 5259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:10.920316 5259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:10.920382 5259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:10.920974 5259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } }
I20260923 22:08:10.921077 5259 leader_election.cc:302] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33f6308ac81a45b1be74a6a080b71924; no voters:
I20260923 22:08:10.921226 5259 leader_election.cc:288] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:10.921278 5264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:10.921423 5259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:10.921435 5264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [term 1 LEADER]: Becoming Leader. State: Replica: 33f6308ac81a45b1be74a6a080b71924, State: Running, Role: LEADER
I20260923 22:08:10.921612 5264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } }
I20260923 22:08:10.921936 5266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33f6308ac81a45b1be74a6a080b71924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } } }
I20260923 22:08:10.921918 5264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33f6308ac81a45b1be74a6a080b71924. Latest consensus state: current_term: 1 leader_uuid: "33f6308ac81a45b1be74a6a080b71924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f6308ac81a45b1be74a6a080b71924" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41655 } } }
I20260923 22:08:10.922000 5266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:10.922008 5264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924 [sys.catalog]: This master's current role is: LEADER
Sep 23 22:08:10 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:10.922366 5270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:10.922880 5270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:10.924325 5270 catalog_manager.cc:1347] Generated new cluster ID: f8dfa45c3d4d4ee6a08338eaacde27ac
I20260923 22:08:10.924365 5270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:10.926252 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:10.916566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33352 (local address 127.1.161.190:41655)
0923 22:08:10.916723 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:10.916730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:10.916760 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:08:10.917138 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:10.917143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:10.917184 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:10.917317 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:10.917957 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:10.918729 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:10.919796 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:10.920212 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:10.920305 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:10.923141 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:10.923145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:10.923146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:10.923157 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:10.924921 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:10.925251 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:10.925254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:10.925254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:10.925336 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:10.925520 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:10.925521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:10.925521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:10.925715 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:10.925850 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:10.925962 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:08:10.926078 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260923 22:08:10.933861 5270 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:10.934275 5270 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:10.936932 5270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33f6308ac81a45b1be74a6a080b71924: Generated new TSK 0
I20260923 22:08:10.937064 5270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:10.961325 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:11.063597 5285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:11.063805 5285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:11.063833 5285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:11.065245 5285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:11.065287 5285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:11.065318 5285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:11.067153 5285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:11.067363 5285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:11.067595 5285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:11.072017 5285 server_base.cc:1034] running on GCE node
W20260923 22:08:11.072106 5293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:11.071982 5291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:11.071970 5290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:11.072531 5285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:11.072768 5285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:11.073920 5285 hybrid_clock.cc:648] HybridClock initialized: now 1790201291073895 us; error 42 us; skew 500 ppm
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:11.076382 5285 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:11.076735 5285 webserver.cc:460] Webserver started at http://127.1.161.129:33733/ using document root <none> and password file <none>
I20260923 22:08:11.076943 5285 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:11.077016 5285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:11.077157 5285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:11.077962 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "96b099a63fb14e399945c127357b5528"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "3901006c504d2f83aaed15f5daa769f4"
server_key_iv: "51e577f000b1c17dc413fc5802846124"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.078255 5285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "96b099a63fb14e399945c127357b5528"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "3901006c504d2f83aaed15f5daa769f4"
server_key_iv: "51e577f000b1c17dc413fc5802846124"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.079397 5285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:11.079962 5300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.080084 5285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.080164 5285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "96b099a63fb14e399945c127357b5528"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "3901006c504d2f83aaed15f5daa769f4"
server_key_iv: "51e577f000b1c17dc413fc5802846124"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.080231 5285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:11.086351 5285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:11.087832 5285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:11.087960 5285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:11.088137 5285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:11.088374 5285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:11.088420 5285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.088452 5285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:11.088485 5285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.093386 5285 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:42781
I20260923 22:08:11.093462 5413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:42781 every 8 connection(s)
I20260923 22:08:11.093719 5285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:11.095675 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5285
I20260923 22:08:11.095767 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:11.095929 1670 external_mini_cluster.cc:1428] Setting key 132b2a467a6705a980c73fdff08d43de
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:11.102263 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.094288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:49737 (local address 127.1.161.190:41655)
0923 22:08:11.094355 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.094357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.094514 (+ 157us) server_negotiation.cc:407] Connection header received
0923 22:08:11.095273 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.095275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.095284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.095335 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:11.095973 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.096509 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.097800 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.098011 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.098040 (+ 29us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.099865 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:11.099866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:11.099867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:11.099873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:11.101065 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.101419 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.101420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.101421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.101468 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.101690 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.101691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.101692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.101793 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:11.101853 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.102082 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.102176 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:11.102458 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.094266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41655 (local address 127.1.161.129:49737)
0923 22:08:11.094411 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.094438 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.095132 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.095379 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.095381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.095644 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:11.095880 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.095885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.096598 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.096599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.097704 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.097706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.098098 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.098098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.098175 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.098602 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:11.098610 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:11.099782 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:11.101156 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.101161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.101163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.101344 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.101536 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.101537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.101537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.101633 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.101855 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:11.101860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:11.101970 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.102064 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.102205 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
I20260923 22:08:11.103863 5414 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41655
I20260923 22:08:11.103940 5414 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:11.104043 5414 heartbeater.cc:511] Master 127.1.161.190:41655 requested a full tablet report, sending...
I20260923 22:08:11.104460 5223 ts_manager.cc:194] Registered new tserver with Master: 96b099a63fb14e399945c127357b5528 (127.1.161.129:42781)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
I20260923 22:08:11.105059 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:49737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:11.117328 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:11.185775 5419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:11.185918 5419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:11.185938 5419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:11.186848 5419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:11.186878 5419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:11.186900 5419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:11.188133 5419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:11.188277 5419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:11.188405 5419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:11.192371 5427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:11.192345 5425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:11.192451 5419 server_base.cc:1034] running on GCE node
W20260923 22:08:11.192354 5424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:11.192787 5419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:11.193045 5419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:11.194190 5419 hybrid_clock.cc:648] HybridClock initialized: now 1790201291194176 us; error 26 us; skew 500 ppm
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:11.196254 5419 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:11.196586 5419 webserver.cc:460] Webserver started at http://127.1.161.130:38739/ using document root <none> and password file <none>
I20260923 22:08:11.196776 5419 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:11.196835 5419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:11.196964 5419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:11.197752 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "36dc7f582818492a8a54f094dc2b1690"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "6331fe547080f5ee0119f2c35faf1f48"
server_key_iv: "00ad279e5169005e31977c44d840a0ca"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.198058 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "36dc7f582818492a8a54f094dc2b1690"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "6331fe547080f5ee0119f2c35faf1f48"
server_key_iv: "00ad279e5169005e31977c44d840a0ca"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.199079 5419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:11.199692 5434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.199821 5419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.199903 5419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "36dc7f582818492a8a54f094dc2b1690"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "6331fe547080f5ee0119f2c35faf1f48"
server_key_iv: "00ad279e5169005e31977c44d840a0ca"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.199975 5419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:11.210031 5419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:11.211867 5419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:11.212004 5419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:11.212182 5419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:11.212466 5419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:11.212518 5419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.212555 5419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:11.212581 5419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.217479 5419 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:39561
I20260923 22:08:11.217509 5547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:39561 every 8 connection(s)
I20260923 22:08:11.217741 5419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:11.220767 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5419
I20260923 22:08:11.220861 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:11.221011 1670 external_mini_cluster.cc:1428] Setting key 491bd47e5aaadfc42b33d8e975853562
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:11.225941 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.218289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:39187 (local address 127.1.161.190:41655)
0923 22:08:11.218326 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.218329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.218511 (+ 182us) server_negotiation.cc:407] Connection header received
0923 22:08:11.219198 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.219199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.219208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.219267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:11.219682 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.220090 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.220866 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.221123 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.221186 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.223480 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:11.223481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:11.223482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:11.223487 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:11.224734 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.225157 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.225158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.225158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.225199 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.225435 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.225435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.225435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.225521 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:11.225562 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.225839 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.225889 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:11.226238 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.218286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41655 (local address 127.1.161.130:39187)
0923 22:08:11.218407 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.218432 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.219099 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.219307 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.219310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.219490 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:11.219627 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.219630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.220155 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.220155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.220774 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.220776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.221201 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.221202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.221313 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.221734 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:11.221742 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:11.223374 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:11.224810 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.224816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.224817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.225086 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.225253 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.225254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.225254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.225375 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.225576 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:11.225582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:11.225722 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.225825 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.225970 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":74,"threads_started":1}
I20260923 22:08:11.226619 5548 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41655
I20260923 22:08:11.226691 5548 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:11.226828 5548 heartbeater.cc:511] Master 127.1.161.190:41655 requested a full tablet report, sending...
I20260923 22:08:11.227135 5223 ts_manager.cc:194] Registered new tserver with Master: 36dc7f582818492a8a54f094dc2b1690 (127.1.161.130:39561)
I20260923 22:08:11.227615 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:39187
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:11.242045 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:11.310237 5553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:11.310381 5553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:11.310400 5553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:11.311396 5553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:11.311431 5553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:11.311453 5553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:11.312785 5553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:11.312999 5553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:11.313220 5553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:11.317670 5553 server_base.cc:1034] running on GCE node
W20260923 22:08:11.317647 5558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:11.317638 5561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:11.317636 5559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:11.318091 5553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:11.318267 5553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:11.319425 5553 hybrid_clock.cc:648] HybridClock initialized: now 1790201291319414 us; error 28 us; skew 500 ppm
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:11.321722 5553 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:11.322135 5553 webserver.cc:460] Webserver started at http://127.1.161.131:40029/ using document root <none> and password file <none>
I20260923 22:08:11.322367 5553 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:11.322408 5553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:11.322538 5553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:11.323621 5553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "630165c624c74259bb23762ae250a498"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "4759bd0596ac21e68674a185eeb305dc"
server_key_iv: "c9d525aeb16acb37d69472bb4feee7de"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.323920 5553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "630165c624c74259bb23762ae250a498"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "4759bd0596ac21e68674a185eeb305dc"
server_key_iv: "c9d525aeb16acb37d69472bb4feee7de"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.325050 5553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:11.325747 5568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.325948 5553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:11.326025 5553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "630165c624c74259bb23762ae250a498"
format_stamp: "Formatted at 2026-09-23 22:08:11 on dist-test-slave-npvd"
server_key: "4759bd0596ac21e68674a185eeb305dc"
server_key_iv: "c9d525aeb16acb37d69472bb4feee7de"
server_key_version: "encryptionkey@0"
I20260923 22:08:11.326103 5553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:11.335994 5553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:11.338083 5553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:11.338171 5553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:11.338361 5553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:11.338624 5553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:11.338676 5553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.338707 5553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:11.338752 5553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:11.343981 5553 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:45297
I20260923 22:08:11.344012 5681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:45297 every 8 connection(s)
I20260923 22:08:11.344273 5553 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:11.345685 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5553
I20260923 22:08:11.345781 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:11.345983 1670 external_mini_cluster.cc:1428] Setting key 6d73972fbc860bccac5e8bafc4992ff6
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:11.351764 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.344830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:41493 (local address 127.1.161.190:41655)
0923 22:08:11.344892 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.344895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.344997 (+ 102us) server_negotiation.cc:407] Connection header received
0923 22:08:11.345821 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.345822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.345833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.345909 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:11.346464 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.346894 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.347796 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.347977 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.348057 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.349657 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:11.349658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:11.349659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:11.349665 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:11.350626 (+ 961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.350949 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.350950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.350950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.351002 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.351234 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.351235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.351235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.351332 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:11.351398 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.351621 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.351689 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:08:11.351886 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.344785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41655 (local address 127.1.161.131:41493)
0923 22:08:11.344907 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:11.344934 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:11.345718 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:11.345966 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:11.345969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:11.346223 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:11.346400 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.346404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.346982 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.346983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.347714 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:11.347716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.348065 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.348066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:11.348179 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.348498 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:11.348504 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:11.349575 (+ 1071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:11.350705 (+ 1130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.350710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.350711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.350885 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.351068 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:11.351069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:11.351069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:11.351154 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:11.351390 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:11.351395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:11.351504 (+ 109us) client_negotiation.cc:769] Sending connection context
0923 22:08:11.351594 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:08:11.351711 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":73,"threads_started":1}
I20260923 22:08:11.352306 5682 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41655
I20260923 22:08:11.352375 5682 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:11.352555 5682 heartbeater.cc:511] Master 127.1.161.190:41655 requested a full tablet report, sending...
I20260923 22:08:11.352887 5223 ts_manager.cc:194] Registered new tserver with Master: 630165c624c74259bb23762ae250a498 (127.1.161.131:45297)
I20260923 22:08:11.353246 5223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:41493
I20260923 22:08:11.357251 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:08:11.376452 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.369921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.1.161.129:42781)
0923 22:08:11.370087 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.370092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.370101 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:11.370129 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.370131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.370142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.370223 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:11.370766 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.371487 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.372110 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.372397 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.372474 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.374313 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:11.374318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:11.374318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:11.374323 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:11.375657 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.375900 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.375902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.375902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.375944 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.376069 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.376069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.376070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.376158 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:11.376239 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.376294 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.376343 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:11.382493 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:11.377388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.1.161.190:41655)
0923 22:08:11.377416 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:11.377418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:11.377443 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:08:11.377498 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:11.377498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:11.377505 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:11.377547 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:11.377901 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.378279 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.378795 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:11.378933 (+ 138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:11.378973 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:11.380700 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:11.380701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:11.380701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:11.380706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:11.381639 (+ 933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.381883 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.381884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.381884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.381921 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:11.382071 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:11.382071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:11.382072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:11.382167 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:11.382265 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:11.382342 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:08:11.382396 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
W20260923 22:08:11.382655 5223 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33364
I20260923 22:08:11.382951 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5285
I20260923 22:08:11.386821 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5419
I20260923 22:08:11.390501 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5553
I20260923 22:08:11.394266 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5191
2026-09-23T22:08:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1639 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5694](info): set up 2 sockets
Sep 23 22:08:11 dist-test-slave-npvd krb5kdc[5694](info): commencing operation
krb5kdc: starting...
W20260923 22:08:12.302536 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.890s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:12.403842 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:34845
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:34845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:12.472114 5707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:12.472262 5707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:12.472281 5707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:12.473377 5707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:12.473415 5707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:12.473429 5707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:12.473441 5707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:12.473493 5707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:12.474753 5707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:34845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:34845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:12.474956 5707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:12.475134 5707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:12.479080 5713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.479087 5715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.479154 5712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.479229 5707 server_base.cc:1034] running on GCE node
I20260923 22:08:12.479449 5707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:12.479640 5707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:12.480795 5707 hybrid_clock.cc:648] HybridClock initialized: now 1790201292480789 us; error 33 us; skew 500 ppm
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:12.483034 5707 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:12.483425 5707 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:12.483493 5707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:12.483621 5707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:12.484495 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "3ef648a6caf8471fa0f48b02488dc6d8"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "e3f9f4faf7e4b75736f2d3507cdc3a7d"
server_key_iv: "12f6b02e13cd91e4c2ba54a37ced841d"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.484794 5707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "3ef648a6caf8471fa0f48b02488dc6d8"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "e3f9f4faf7e4b75736f2d3507cdc3a7d"
server_key_iv: "12f6b02e13cd91e4c2ba54a37ced841d"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.485816 5707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:12.486472 5721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.486641 5707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:12.486733 5707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "3ef648a6caf8471fa0f48b02488dc6d8"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "e3f9f4faf7e4b75736f2d3507cdc3a7d"
server_key_iv: "12f6b02e13cd91e4c2ba54a37ced841d"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.486809 5707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:12.524005 5707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:12.525810 5707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:12.525979 5707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:12.529276 5707 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:34845
I20260923 22:08:12.529300 5773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:34845 every 8 connection(s)
I20260923 22:08:12.529510 5707 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:12.530207 5774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:12.531989 5774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: Bootstrap starting.
I20260923 22:08:12.532526 5774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:12.532823 5774 log.cc:824] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:12.533443 5774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: No bootstrap required, opened a new log
I20260923 22:08:12.534771 5774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } }
I20260923 22:08:12.534876 5774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:12.534914 5774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef648a6caf8471fa0f48b02488dc6d8, State: Initialized, Role: FOLLOWER
I20260923 22:08:12.535035 5774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } }
I20260923 22:08:12.535085 5774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:12.535104 5774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:12.535158 5774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:12.535825 5774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } }
I20260923 22:08:12.535897 5774 leader_election.cc:302] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ef648a6caf8471fa0f48b02488dc6d8; no voters:
I20260923 22:08:12.536027 5774 leader_election.cc:288] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:12.536128 5777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:12.536244 5774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:12.536312 5777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef648a6caf8471fa0f48b02488dc6d8, State: Running, Role: LEADER
I20260923 22:08:12.536396 5777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } }
I20260923 22:08:12.536721 5777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ef648a6caf8471fa0f48b02488dc6d8. Latest consensus state: current_term: 1 leader_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } } }
I20260923 22:08:12.536784 5777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:12.536836 5778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ef648a6caf8471fa0f48b02488dc6d8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 34845 } } }
I20260923 22:08:12.536976 5786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:12.537544 5786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:12.537065 5778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:12.537603 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5707
I20260923 22:08:12.537694 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:12.537861 1670 external_mini_cluster.cc:1428] Setting key c9d3ded0ddce9d7d1cd8f97a56f61057
I20260923 22:08:12.539274 5786 catalog_manager.cc:1347] Generated new cluster ID: 2da5796c4dcb4d3abd260fc10ebd9fd6
I20260923 22:08:12.539330 5786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:12.544505 5786 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:12.545094 5786 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:12.546828 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.538372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.1.161.190:34845)
0923 22:08:12.538556 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.538563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.538591 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:08:12.538693 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.538699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.538759 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.538886 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.539476 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.540815 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.541593 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.542000 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.542117 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.544155 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.544158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.544159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.544168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.545598 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.545906 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.545908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.545909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.545997 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.546173 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.546173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.546174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.546349 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.546502 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.546557 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.546675 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":61,"threads_started":1}
I20260923 22:08:12.556783 5786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: Generated new TSK 0
I20260923 22:08:12.556911 5786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:12.584107 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:12.658249 5800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:12.658427 5800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:12.658459 5800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:12.659485 5800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:12.659534 5800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:12.659571 5800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:12.660837 5800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:12.661051 5800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:12.661231 5800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:12.666764 5806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.666797 5805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.666941 5808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.667563 5800 server_base.cc:1034] running on GCE node
I20260923 22:08:12.667757 5800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:12.668015 5800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:12.669135 5800 hybrid_clock.cc:648] HybridClock initialized: now 1790201292669121 us; error 33 us; skew 500 ppm
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:12.672168 5800 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:12.672457 5800 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:12.672498 5800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:12.672628 5800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:12.673635 5800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "3d295c85b3724034a0bc70ec54119414"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "84fba5474de4fa70fb6036b39f49536d"
server_key_iv: "b1beeb8f1542cf7794ed41ec34a808c0"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.673980 5800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "3d295c85b3724034a0bc70ec54119414"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "84fba5474de4fa70fb6036b39f49536d"
server_key_iv: "b1beeb8f1542cf7794ed41ec34a808c0"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.675207 5800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:12.675921 5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.676214 5800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:12.676311 5800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "3d295c85b3724034a0bc70ec54119414"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "84fba5474de4fa70fb6036b39f49536d"
server_key_iv: "b1beeb8f1542cf7794ed41ec34a808c0"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.676389 5800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:12.692592 5800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:12.694255 5800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:12.694387 5800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:12.694571 5800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:12.694823 5800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:12.694868 5800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.694895 5800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:12.694919 5800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.699955 5800 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:40141
I20260923 22:08:12.699991 5927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:40141 every 8 connection(s)
I20260923 22:08:12.700193 5800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:12.707291 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.700687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:37557 (local address 127.1.161.190:34845)
0923 22:08:12.700752 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.700754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.700920 (+ 166us) server_negotiation.cc:407] Connection header received
0923 22:08:12.701324 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.701325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.701333 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.701379 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.702057 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.702475 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.703463 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.703649 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.703701 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.705320 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.705321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.705321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.705326 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.706327 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.706616 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.706617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.706617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.706657 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.706826 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.706827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.706827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.706904 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.706945 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.707125 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.707233 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:12.707384 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.700669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:34845 (local address 127.1.161.129:37557)
0923 22:08:12.700808 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:12.700841 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:12.701247 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:12.701428 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:12.701430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:12.701623 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:12.701997 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.702001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.702540 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.702540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.703397 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.703399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.703717 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.703717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.703819 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.704167 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:12.704173 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:12.705246 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:12.706390 (+ 1144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.706395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.706397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.706563 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.706706 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.706707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.706707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.706783 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.706948 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:12.706951 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:12.707033 (+ 82us) client_negotiation.cc:769] Sending connection context
0923 22:08:12.707097 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:08:12.707235 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260923 22:08:12.707798 5928 heartbeater.cc:347] Connected to a master server at 127.1.161.190:34845
I20260923 22:08:12.707875 5928 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:12.708002 5928 heartbeater.cc:511] Master 127.1.161.190:34845 requested a full tablet report, sending...
I20260923 22:08:12.707996 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5800
I20260923 22:08:12.708087 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:12.708240 1670 external_mini_cluster.cc:1428] Setting key aed18f6d67ced05ad14a1c99b5637947
I20260923 22:08:12.708483 5738 ts_manager.cc:194] Registered new tserver with Master: 3d295c85b3724034a0bc70ec54119414 (127.1.161.129:40141)
I20260923 22:08:12.709100 5738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:37557
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:12.729646 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:12.800215 5933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:12.800375 5933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:12.800396 5933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:12.801434 5933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:12.801470 5933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:12.801493 5933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:12.802825 5933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:12.803036 5933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:12.803275 5933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:12.807720 5938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.807757 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.807730 5939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.807857 5933 server_base.cc:1034] running on GCE node
I20260923 22:08:12.808182 5933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:12.808423 5933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:12.809579 5933 hybrid_clock.cc:648] HybridClock initialized: now 1790201292809568 us; error 23 us; skew 500 ppm
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:12.811802 5933 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:12.812170 5933 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:12.812265 5933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:12.812409 5933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:12.813462 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "bec7e15c85b84a4bb759e6012519e5e1"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "0afc38ddc444c41ba75be30145fbaefd"
server_key_iv: "cbbacbf783bab3062efb0f141842133a"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.813824 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "bec7e15c85b84a4bb759e6012519e5e1"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "0afc38ddc444c41ba75be30145fbaefd"
server_key_iv: "cbbacbf783bab3062efb0f141842133a"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.815110 5933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:12.815850 5947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.815999 5933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:12.816099 5933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "bec7e15c85b84a4bb759e6012519e5e1"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "0afc38ddc444c41ba75be30145fbaefd"
server_key_iv: "cbbacbf783bab3062efb0f141842133a"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.816169 5933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:12.826941 5933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:12.828442 5933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:12.828581 5933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:12.828773 5933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:12.829030 5933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:12.829088 5933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.829123 5933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:12.829146 5933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.834373 5933 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:34387
I20260923 22:08:12.834403 6060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:34387 every 8 connection(s)
I20260923 22:08:12.834628 5933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:12.842962 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.835390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:41753 (local address 127.1.161.190:34845)
0923 22:08:12.835462 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.835464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.835610 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:08:12.836101 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.836103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.836115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.836160 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.837067 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.837533 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.838546 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.838766 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.838838 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.840654 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.840655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.840656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.840665 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.841701 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.842067 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.842068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.842069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.842123 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.842350 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.842351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.842351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.842458 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.842522 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.842795 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.842871 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:12.843124 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.835300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:34845 (local address 127.1.161.130:41753)
0923 22:08:12.835474 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:12.835537 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:12.835983 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:12.836215 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:12.836218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:12.836486 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:12.836972 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.836977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.837636 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.837637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.838446 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.838449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.838843 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.838844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.838957 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.839334 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:12.839341 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:12.840558 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:12.841788 (+ 1230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.841794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.841796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.841981 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.842182 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.842183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.842183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.842281 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.842526 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:12.842532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:12.842664 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 22:08:12.842774 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 22:08:12.842912 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260923 22:08:12.843529 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 5933
I20260923 22:08:12.843653 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:12.843672 6061 heartbeater.cc:347] Connected to a master server at 127.1.161.190:34845
I20260923 22:08:12.843761 6061 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:12.843899 1670 external_mini_cluster.cc:1428] Setting key 20d612f7ee6eee318d71c92b6fd184d7
I20260923 22:08:12.843953 6061 heartbeater.cc:511] Master 127.1.161.190:34845 requested a full tablet report, sending...
I20260923 22:08:12.844363 5737 ts_manager.cc:194] Registered new tserver with Master: bec7e15c85b84a4bb759e6012519e5e1 (127.1.161.130:34387)
I20260923 22:08:12.844738 5737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:41753
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:12.865701 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:12.934408 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:12.934587 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:12.934616 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:12.935667 6066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:12.935729 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:12.935773 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:12.937129 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:34845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:12.937341 6066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:12.937526 6066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:12.941597 6071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.941628 6066 server_base.cc:1034] running on GCE node
W20260923 22:08:12.941597 6074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:12.941627 6072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:12.942052 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:12.942258 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:12.943418 6066 hybrid_clock.cc:648] HybridClock initialized: now 1790201292943414 us; error 30 us; skew 500 ppm
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:12.945581 6066 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:12.945881 6066 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:12.945922 6066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:12.945995 6066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:12.946744 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "4e94ae401a854a4ca6211cb0ff214c47"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "d1029543dfb036818d3f795ea9f3568d"
server_key_iv: "d42df7fadfeafd78b5941f8fdc748dcf"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.946978 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "4e94ae401a854a4ca6211cb0ff214c47"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "d1029543dfb036818d3f795ea9f3568d"
server_key_iv: "d42df7fadfeafd78b5941f8fdc748dcf"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.948014 6066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:12.948572 6080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.948721 6066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:12.948807 6066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "4e94ae401a854a4ca6211cb0ff214c47"
format_stamp: "Formatted at 2026-09-23 22:08:12 on dist-test-slave-npvd"
server_key: "d1029543dfb036818d3f795ea9f3568d"
server_key_iv: "d42df7fadfeafd78b5941f8fdc748dcf"
server_key_version: "encryptionkey@0"
I20260923 22:08:12.948875 6066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:12.971258 6066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:12.973219 6066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:12.973380 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:12.973580 6066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:12.973855 6066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:12.973923 6066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.973958 6066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:12.973984 6066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:12.978777 6066 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:46835
I20260923 22:08:12.978817 6193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:46835 every 8 connection(s)
I20260923 22:08:12.979007 6066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:12.979900 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6066
I20260923 22:08:12.980001 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:12.980180 1670 external_mini_cluster.cc:1428] Setting key fb28bf69f59a1caba715537483d97ca7
Sep 23 22:08:12 dist-test-slave-npvd krb5kdc[5694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201292, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:12.986418 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.979602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:55735 (local address 127.1.161.190:34845)
0923 22:08:12.979675 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.979677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.979823 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:08:12.980379 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.980395 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.980406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.980460 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.981322 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.981730 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.982489 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.982644 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.982702 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.984417 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.984418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.984419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.984424 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.985569 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.985845 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.985846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.985846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.985887 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.986045 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.986045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.986046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.986120 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.986156 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.986296 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.986349 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:12.986589 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.979568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:34845 (local address 127.1.161.131:55735)
0923 22:08:12.979714 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:12.979741 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:12.980280 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:12.980533 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:12.980537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:12.980808 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:12.981248 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.981253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.981793 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.981793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.982428 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:12.982429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.982720 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.982721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:12.982815 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.983181 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:12.983187 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:12.984344 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:12.985625 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.985628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.985629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.985790 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.985934 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:12.985934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:12.985935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:12.986006 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:12.986149 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:12.986151 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:12.986228 (+ 77us) client_negotiation.cc:769] Sending connection context
0923 22:08:12.986285 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:08:12.986372 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260923 22:08:12.986946 6194 heartbeater.cc:347] Connected to a master server at 127.1.161.190:34845
I20260923 22:08:12.987007 6194 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:12.987088 6194 heartbeater.cc:511] Master 127.1.161.190:34845 requested a full tablet report, sending...
I20260923 22:08:12.987385 5737 ts_manager.cc:194] Registered new tserver with Master: 4e94ae401a854a4ca6211cb0ff214c47 (127.1.161.131:46835)
I20260923 22:08:12.987695 5737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:55735
I20260923 22:08:12.991595 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:12.997896 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:12.992663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.1.161.190:34845)
0923 22:08:12.992763 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:12.992765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:12.992859 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:08:12.992931 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:12.992932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:12.992942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:12.992996 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:12.993552 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.993992 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.994627 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:12.994798 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:12.994860 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:12.995797 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:12.995799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:12.995799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:12.995805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:12.996872 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.997213 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.997214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.997214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.997277 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:12.997467 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:12.997468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:12.997468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:12.997583 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:12.997670 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:12.997756 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:08:12.997816 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 22:08:12.999337 5737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:13.000659 5737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:13.005924 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.1.161.129:40141)
0923 22:08:13.002638 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.002642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.002695 (+ 53us) server_negotiation.cc:407] Connection header received
0923 22:08:13.002758 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.002761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.002775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.002861 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.002976 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.004138 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005062 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005545 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005617 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.005665 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.005795 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.005842 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":75,"threads_started":1}
I20260923 22:08:13.005925 6204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46835 (local address 127.0.0.1:48112)
0923 22:08:13.002472 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.002508 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.002565 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.002825 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.002828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.002830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.002888 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.002890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.003941 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.003942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.004933 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.004935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005698 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.005752 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.005766 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.005809 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.005866 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":52,"threads_started":1}
I20260923 22:08:13.006161 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40141 (local address 127.0.0.1:55572)
0923 22:08:13.002646 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.002652 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.002688 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.002869 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.002871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.002873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.002920 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.002921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.004233 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.004234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.004990 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.004992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005627 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.005693 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.005705 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.006038 (+ 333us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.006090 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":24,"thread_start_us":58,"threads_started":1}
I20260923 22:08:13.006177 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.1.161.131:46835)
0923 22:08:13.002561 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.002564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.002574 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:13.002632 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.002633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.002647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.002741 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.002963 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.003840 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005010 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005571 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005628 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.005668 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.006084 (+ 416us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.006119 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":57,"threads_started":1}
I20260923 22:08:13.006404 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.1.161.130:34387)
0923 22:08:13.002725 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.002728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.002738 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:13.002773 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.002775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.002790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.002872 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.003059 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.004129 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005234 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005833 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005881 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.005921 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.006126 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.006161 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":65,"threads_started":1}
I20260923 22:08:13.006326 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.002289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:34387 (local address 127.0.0.1:58344)
0923 22:08:13.002511 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.002518 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.002565 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.002959 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.002960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.002962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.002996 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.002997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.004226 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.004227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.005164 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.005166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.005948 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.005949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.005999 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.006003 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.006045 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.006080 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":24,"thread_start_us":45,"threads_started":1}
I20260923 22:08:13.006705 6128 tablet_service.cc:1467] Processing CreateTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 (DEFAULT_TABLE table=test-table [id=1c3478baadae443da4fb829e1872ba35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.007000 6128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394501499eff49b5aa5c8e3fdbfa2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.006923 5862 tablet_service.cc:1467] Processing CreateTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 (DEFAULT_TABLE table=test-table [id=1c3478baadae443da4fb829e1872ba35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.007082 5995 tablet_service.cc:1467] Processing CreateTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 (DEFAULT_TABLE table=test-table [id=1c3478baadae443da4fb829e1872ba35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.007223 5862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394501499eff49b5aa5c8e3fdbfa2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.007361 5995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394501499eff49b5aa5c8e3fdbfa2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.009689 6208 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: Bootstrap starting.
I20260923 22:08:13.009688 6209 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Bootstrap starting.
I20260923 22:08:13.010283 6209 tablet_bootstrap.cc:654] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.010371 6208 tablet_bootstrap.cc:654] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.010880 6209 log.cc:824] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.011255 6208 log.cc:824] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.011629 6209 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: No bootstrap required, opened a new log
I20260923 22:08:13.011687 6209 ts_tablet_manager.cc:1397] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:13.011873 6208 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: No bootstrap required, opened a new log
I20260923 22:08:13.011920 6208 ts_tablet_manager.cc:1397] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:08:13.012171 6212 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Bootstrap starting.
I20260923 22:08:13.012549 6212 tablet_bootstrap.cc:654] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.012970 6212 log.cc:824] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.013449 6209 raft_consensus.cc:348] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.013595 6212 tablet_bootstrap.cc:492] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: No bootstrap required, opened a new log
I20260923 22:08:13.013597 6209 raft_consensus.cc:374] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.013634 6212 ts_tablet_manager.cc:1397] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:13.013631 6209 raft_consensus.cc:729] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e94ae401a854a4ca6211cb0ff214c47, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.013742 6209 consensus_queue.cc:260] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.013705 6208 raft_consensus.cc:348] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.013839 6208 raft_consensus.cc:374] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.013877 6208 raft_consensus.cc:729] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d295c85b3724034a0bc70ec54119414, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.014007 6209 ts_tablet_manager.cc:1428] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:13.013988 6208 consensus_queue.cc:260] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.014020 6194 heartbeater.cc:503] Master 127.1.161.190:34845 was elected leader, sending a full tablet report...
I20260923 22:08:13.014248 6208 ts_tablet_manager.cc:1428] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:08:13.014369 5928 heartbeater.cc:503] Master 127.1.161.190:34845 was elected leader, sending a full tablet report...
I20260923 22:08:13.015731 6212 raft_consensus.cc:348] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.015836 6212 raft_consensus.cc:374] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.015853 6212 raft_consensus.cc:729] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec7e15c85b84a4bb759e6012519e5e1, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.015942 6212 consensus_queue.cc:260] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.016129 6212 ts_tablet_manager.cc:1428] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:13.016180 6061 heartbeater.cc:503] Master 127.1.161.190:34845 was elected leader, sending a full tablet report...
I20260923 22:08:13.040588 6215 raft_consensus.cc:482] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:13.040671 6215 raft_consensus.cc:504] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.040907 6215 leader_election.cc:288] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e94ae401a854a4ca6211cb0ff214c47 (127.1.161.131:46835), bec7e15c85b84a4bb759e6012519e5e1 (127.1.161.130:34387)
I20260923 22:08:13.043365 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.041028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:34387 (local address 127.1.161.129:33987)
0923 22:08:13.041073 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.041085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.041124 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.041293 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.041294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.041295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.041335 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.041336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.041871 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.041872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.042556 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.042558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.043109 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.043110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.043197 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.043200 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.043249 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.043289 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:08:13.043370 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.041148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:33987 (local address 127.1.161.130:34387)
0923 22:08:13.041186 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.041188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.041195 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:13.041213 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.041214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.041222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.041260 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.041402 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.041812 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.042631 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.043038 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.043090 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.043095 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.043254 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.043289 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:13.043467 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.041028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46835 (local address 127.1.161.129:57909)
0923 22:08:13.041162 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.041171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.041224 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.041394 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.041395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.041397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:13.041452 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.041453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.041955 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.041956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.042628 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:13.042630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.043210 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.043211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:13.043286 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.043290 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.043346 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.043380 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":24,"thread_start_us":60,"threads_started":1}
I20260923 22:08:13.043469 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.041106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:57909 (local address 127.1.161.131:46835)
0923 22:08:13.041185 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.041187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.041214 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:13.041302 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.041302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.041309 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.041356 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:13.041508 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.041873 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.042719 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.043062 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.043147 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.043153 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.043362 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.043401 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 22:08:13.043684 6015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394501499eff49b5aa5c8e3fdbfa2dc0" candidate_uuid: "3d295c85b3724034a0bc70ec54119414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec7e15c85b84a4bb759e6012519e5e1" is_pre_election: true
I20260923 22:08:13.043813 6015 raft_consensus.cc:2454] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d295c85b3724034a0bc70ec54119414 in term 0.
I20260923 22:08:13.043841 6148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394501499eff49b5aa5c8e3fdbfa2dc0" candidate_uuid: "3d295c85b3724034a0bc70ec54119414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e94ae401a854a4ca6211cb0ff214c47" is_pre_election: true
I20260923 22:08:13.043954 6148 raft_consensus.cc:2454] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d295c85b3724034a0bc70ec54119414 in term 0.
I20260923 22:08:13.044015 5815 leader_election.cc:302] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d295c85b3724034a0bc70ec54119414, bec7e15c85b84a4bb759e6012519e5e1; no voters:
I20260923 22:08:13.044122 6215 raft_consensus.cc:2781] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:13.044173 6215 raft_consensus.cc:482] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:13.044188 6215 raft_consensus.cc:3037] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.044672 6215 raft_consensus.cc:504] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.044777 6215 leader_election.cc:288] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [CANDIDATE]: Term 1 election: Requested vote from peers 4e94ae401a854a4ca6211cb0ff214c47 (127.1.161.131:46835), bec7e15c85b84a4bb759e6012519e5e1 (127.1.161.130:34387)
I20260923 22:08:13.044927 6148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394501499eff49b5aa5c8e3fdbfa2dc0" candidate_uuid: "3d295c85b3724034a0bc70ec54119414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e94ae401a854a4ca6211cb0ff214c47"
I20260923 22:08:13.044927 6015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394501499eff49b5aa5c8e3fdbfa2dc0" candidate_uuid: "3d295c85b3724034a0bc70ec54119414" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec7e15c85b84a4bb759e6012519e5e1"
I20260923 22:08:13.045002 6015 raft_consensus.cc:3037] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.045002 6148 raft_consensus.cc:3037] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.045755 6015 raft_consensus.cc:2454] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d295c85b3724034a0bc70ec54119414 in term 1.
I20260923 22:08:13.045765 6148 raft_consensus.cc:2454] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d295c85b3724034a0bc70ec54119414 in term 1.
I20260923 22:08:13.045928 5818 leader_election.cc:302] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d295c85b3724034a0bc70ec54119414, 4e94ae401a854a4ca6211cb0ff214c47; no voters:
I20260923 22:08:13.046046 6215 raft_consensus.cc:2781] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:13.046168 6215 raft_consensus.cc:686] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [term 1 LEADER]: Becoming Leader. State: Replica: 3d295c85b3724034a0bc70ec54119414, State: Running, Role: LEADER
I20260923 22:08:13.046242 6215 consensus_queue.cc:237] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } }
I20260923 22:08:13.046854 5737 catalog_manager.cc:5495] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d295c85b3724034a0bc70ec54119414 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "3d295c85b3724034a0bc70ec54119414" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d295c85b3724034a0bc70ec54119414" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40141 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:13.050860 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.049115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.1.161.129:40141)
0923 22:08:13.049145 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.049147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.049152 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:08:13.049227 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.049228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.049238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.049283 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:13.049362 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.049699 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.050301 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:13.050445 (+ 144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:13.050485 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:13.050596 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:13.050660 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:13.050727 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.050759 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.050771 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:13.052132 6015 raft_consensus.cc:1261] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 1 FOLLOWER]: Refusing update from remote peer 3d295c85b3724034a0bc70ec54119414: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.052146 6148 raft_consensus.cc:1261] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 1 FOLLOWER]: Refusing update from remote peer 3d295c85b3724034a0bc70ec54119414: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.052386 6215 consensus_queue.cc:1035] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e94ae401a854a4ca6211cb0ff214c47" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.052496 6218 consensus_queue.cc:1035] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec7e15c85b84a4bb759e6012519e5e1" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 34387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.054613 6221 mvcc.cc:205] Tried to move back new op lower bound from 7332664496339206144 to 7332664496317493248. Current Snapshot: MvccSnapshot[applied={T|T < 7332664496339206144}]
I20260923 22:08:13.056632 5738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.056716 5738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.057255 5738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ef648a6caf8471fa0f48b02488dc6d8: Sending DeleteTablet for 3 replicas of tablet 394501499eff49b5aa5c8e3fdbfa2dc0
I20260923 22:08:13.057533 6128 tablet_service.cc:1514] Processing DeleteTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:48112
I20260923 22:08:13.057544 5995 tablet_service.cc:1514] Processing DeleteTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:58344
I20260923 22:08:13.057705 5862 tablet_service.cc:1514] Processing DeleteTablet for tablet 394501499eff49b5aa5c8e3fdbfa2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:55572
I20260923 22:08:13.057866 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5800
I20260923 22:08:13.057873 6233 tablet_replica.cc:330] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 3d295c85b3724034a0bc70ec54119414: stopping tablet replica
I20260923 22:08:13.057886 6232 tablet_replica.cc:330] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: stopping tablet replica
I20260923 22:08:13.058436 6232 raft_consensus.cc:2229] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.058552 6232 raft_consensus.cc:2258] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.058696 6231 tablet_replica.cc:330] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: stopping tablet replica
I20260923 22:08:13.058809 6231 raft_consensus.cc:2229] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.058924 6231 raft_consensus.cc:2258] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.059336 6231 ts_tablet_manager.cc:1905] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.059569 6232 ts_tablet_manager.cc:1905] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.060803 6231 ts_tablet_manager.cc:1918] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.060861 6231 log.cc:1196] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/394501499eff49b5aa5c8e3fdbfa2dc0
I20260923 22:08:13.060986 6232 ts_tablet_manager.cc:1918] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.061074 6232 log.cc:1196] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/394501499eff49b5aa5c8e3fdbfa2dc0
I20260923 22:08:13.061115 6231 ts_tablet_manager.cc:1939] T 394501499eff49b5aa5c8e3fdbfa2dc0 P 4e94ae401a854a4ca6211cb0ff214c47: Deleting consensus metadata
I20260923 22:08:13.061343 6232 ts_tablet_manager.cc:1939] T 394501499eff49b5aa5c8e3fdbfa2dc0 P bec7e15c85b84a4bb759e6012519e5e1: Deleting consensus metadata
I20260923 22:08:13.061439 5725 catalog_manager.cc:4841] TS 4e94ae401a854a4ca6211cb0ff214c47 (127.1.161.131:46835): tablet 394501499eff49b5aa5c8e3fdbfa2dc0 (table test-table [id=1c3478baadae443da4fb829e1872ba35]) successfully deleted
I20260923 22:08:13.061685 5722 catalog_manager.cc:4841] TS bec7e15c85b84a4bb759e6012519e5e1 (127.1.161.130:34387): tablet 394501499eff49b5aa5c8e3fdbfa2dc0 (table test-table [id=1c3478baadae443da4fb829e1872ba35]) successfully deleted
W20260923 22:08:13.063346 5723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:40141 (error 108)
I20260923 22:08:13.063526 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5933
I20260923 22:08:13.064352 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.064229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40141 (local address 127.0.0.1:55592)
0923 22:08:13.064289 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.064324 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 22:08:13.064474 5723 catalog_manager.cc:4584] TS 3d295c85b3724034a0bc70ec54119414 (127.1.161.129:40141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 394501499eff49b5aa5c8e3fdbfa2dc0: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40141: connect: Connection refused (error 111)
I20260923 22:08:13.067730 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6066
I20260923 22:08:13.071614 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 5707
2026-09-23T22:08:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1678 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T22:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:13Z Disabled control of system clock
I20260923 22:08:13.096971 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41747
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:08:13.167104 6238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.167272 6238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.167290 6238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.168176 6238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:13.168213 6238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.168224 6238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.168236 6238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:13.168246 6238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:13.169279 6238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:13.169425 6238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.169628 6238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.173868 6246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.173856 6243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.173852 6244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.173889 6238 server_base.cc:1034] running on GCE node
I20260923 22:08:13.174258 6238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.174520 6238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.175669 6238 hybrid_clock.cc:648] HybridClock initialized: now 1790201293175651 us; error 28 us; skew 500 ppm
I20260923 22:08:13.176651 6238 webserver.cc:460] Webserver started at http://127.1.161.190:37139/ using document root <none> and password file <none>
I20260923 22:08:13.176853 6238 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.176913 6238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.177021 6238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.177903 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "092f7609c23f486986bdf9832dd637c6"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "4b3615aaf0c772f79e8bc169782720d5"
server_key_iv: "772392b34c8d1f4ec467abd464725415"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.178229 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "092f7609c23f486986bdf9832dd637c6"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "4b3615aaf0c772f79e8bc169782720d5"
server_key_iv: "772392b34c8d1f4ec467abd464725415"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.179389 6238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:13.179984 6252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.180133 6238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.180197 6238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "092f7609c23f486986bdf9832dd637c6"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "4b3615aaf0c772f79e8bc169782720d5"
server_key_iv: "772392b34c8d1f4ec467abd464725415"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.180269 6238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.182823 6238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.183053 6238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.183198 6238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.186285 6238 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41747
I20260923 22:08:13.186349 6304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41747 every 8 connection(s)
I20260923 22:08:13.186582 6238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:13.187115 6305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.189163 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: Bootstrap starting.
I20260923 22:08:13.189625 6305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.189888 6305 log.cc:824] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.190294 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6238
I20260923 22:08:13.190397 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:13.190486 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: No bootstrap required, opened a new log
I20260923 22:08:13.190572 1670 external_mini_cluster.cc:1428] Setting key 611c3f80daed58ddb4a1eb43520d0aff
I20260923 22:08:13.192482 6305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } }
I20260923 22:08:13.192600 6305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.192623 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.190972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.1.161.190:41747)
0923 22:08:13.191158 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.191192 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.191217 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:08:13.191597 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.191602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.191630 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.191718 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.192232 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.192235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.192235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.192238 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.192268 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.192276 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.192320 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.192392 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.192447 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260923 22:08:13.192618 6305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092f7609c23f486986bdf9832dd637c6, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.192740 6305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } }
I20260923 22:08:13.192862 6305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:13.192894 6305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:13.192940 6305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.193729 6305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } }
I20260923 22:08:13.193841 6305 leader_election.cc:302] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092f7609c23f486986bdf9832dd637c6; no voters:
I20260923 22:08:13.194003 6305 leader_election.cc:288] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:13.194052 6310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:13.194173 6310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [term 1 LEADER]: Becoming Leader. State: Replica: 092f7609c23f486986bdf9832dd637c6, State: Running, Role: LEADER
I20260923 22:08:13.194208 6305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:13.194289 6310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } }
I20260923 22:08:13.194676 6312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092f7609c23f486986bdf9832dd637c6. Latest consensus state: current_term: 1 leader_uuid: "092f7609c23f486986bdf9832dd637c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } } }
I20260923 22:08:13.194765 6312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:13.194908 6311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "092f7609c23f486986bdf9832dd637c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092f7609c23f486986bdf9832dd637c6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41747 } } }
I20260923 22:08:13.194924 6319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:13.194953 6311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:13.195350 6319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:13.196619 6319 catalog_manager.cc:1347] Generated new cluster ID: 9bc316393f1646a895186d8b23ca3bef
I20260923 22:08:13.196678 6319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:13.205324 6319 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:13.205464 6319 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:13.214447 6319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: Generated new TSK 0
I20260923 22:08:13.214584 6319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:13.215857 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--builtin_ntp_servers=127.1.161.148:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 22:08:13.283743 6329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.283954 6329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.284010 6329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.285001 6329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.285058 6329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.285101 6329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:13.286356 6329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:13.286567 6329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.286832 6329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.291939 6335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.292004 6329 server_base.cc:1034] running on GCE node
W20260923 22:08:13.291903 6334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.291913 6337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.292224 6329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.292448 6329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.293594 6329 hybrid_clock.cc:648] HybridClock initialized: now 1790201293293572 us; error 37 us; skew 500 ppm
I20260923 22:08:13.294777 6329 webserver.cc:460] Webserver started at http://127.1.161.129:34991/ using document root <none> and password file <none>
I20260923 22:08:13.295042 6329 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.295083 6329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.295212 6329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.296111 6329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "436c6530c8b84590aa11581f102ca88c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "7b71528909a4210236423525491b0d0e"
server_key_iv: "252dd91b954caae4e8897fc34a3423af"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.296423 6329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "436c6530c8b84590aa11581f102ca88c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "7b71528909a4210236423525491b0d0e"
server_key_iv: "252dd91b954caae4e8897fc34a3423af"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.297580 6329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:13.298305 6343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.298460 6329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:13.298556 6329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "436c6530c8b84590aa11581f102ca88c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "7b71528909a4210236423525491b0d0e"
server_key_iv: "252dd91b954caae4e8897fc34a3423af"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.298621 6329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.301472 6329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.301687 6329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.301796 6329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.301995 6329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:13.302274 6329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.302309 6329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.302326 6329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.302357 6329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.307468 6329 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46687
I20260923 22:08:13.307500 6456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46687 every 8 connection(s)
I20260923 22:08:13.307750 6329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:13.309662 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6329
I20260923 22:08:13.309729 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:13.309891 1670 external_mini_cluster.cc:1428] Setting key 515b78a3238e0b281c681f0f63312724
I20260923 22:08:13.310086 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.308330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:45827 (local address 127.1.161.190:41747)
0923 22:08:13.308400 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.308402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.308546 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:08:13.308887 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.308888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.308892 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.308935 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.309837 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.309838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.309838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.309843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.309856 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.309860 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.309926 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.310016 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.310031 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:13.310237 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.308293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41747 (local address 127.1.161.129:45827)
0923 22:08:13.308468 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.308492 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.308815 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.308973 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.308976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.308982 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.309661 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.309662 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.309682 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.309725 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.309924 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.309930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.309932 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.310016 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.310066 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260923 22:08:13.310598 6457 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41747
I20260923 22:08:13.310671 6457 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.310678 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--builtin_ntp_servers=127.1.161.148:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 22:08:13.310808 6457 heartbeater.cc:511] Master 127.1.161.190:41747 requested a full tablet report, sending...
I20260923 22:08:13.311267 6269 ts_manager.cc:194] Registered new tserver with Master: 436c6530c8b84590aa11581f102ca88c (127.1.161.129:46687)
W20260923 22:08:13.385444 6460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.385689 6460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.385731 6460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.387287 6460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.387375 6460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.387427 6460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:13.388787 6460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:13.389086 6460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.389389 6460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.395370 6468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.395401 6466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.395540 6460 server_base.cc:1034] running on GCE node
W20260923 22:08:13.395421 6465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.395771 6460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.396003 6460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.397145 6460 hybrid_clock.cc:648] HybridClock initialized: now 1790201293397125 us; error 33 us; skew 500 ppm
I20260923 22:08:13.398430 6460 webserver.cc:460] Webserver started at http://127.1.161.130:36021/ using document root <none> and password file <none>
I20260923 22:08:13.398639 6460 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.398684 6460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.398820 6460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.399832 6460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "33f05d74011b44e08b0363bea26b0b98"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "2cb3a9aa0aa6c76983d82a6fd59e29dc"
server_key_iv: "04edd0b341c3fdae3ccfdb68160c663d"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.400166 6460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "33f05d74011b44e08b0363bea26b0b98"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "2cb3a9aa0aa6c76983d82a6fd59e29dc"
server_key_iv: "04edd0b341c3fdae3ccfdb68160c663d"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.401458 6460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:13.402160 6474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.402311 6460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.402388 6460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "33f05d74011b44e08b0363bea26b0b98"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "2cb3a9aa0aa6c76983d82a6fd59e29dc"
server_key_iv: "04edd0b341c3fdae3ccfdb68160c663d"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.402458 6460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.405808 6460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.406082 6460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.406229 6460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.406457 6460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:13.406733 6460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.406790 6460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.406829 6460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.406849 6460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.412482 6460 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:41685
I20260923 22:08:13.412508 6587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:41685 every 8 connection(s)
I20260923 22:08:13.412917 6460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:13.414870 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6460
I20260923 22:08:13.414994 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:13.415252 1670 external_mini_cluster.cc:1428] Setting key 06998380208ced43a9f20045ffb403f6
I20260923 22:08:13.415781 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.413557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:57483 (local address 127.1.161.190:41747)
0923 22:08:13.413614 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.413616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.413814 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:08:13.414219 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.414220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.414224 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.414267 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.415523 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.415525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.415525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.415530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.415546 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.415551 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.415608 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.415694 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.415706 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:13.415966 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.413556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41747 (local address 127.1.161.130:57483)
0923 22:08:13.413702 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.413730 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.414133 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.414330 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.414334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.414340 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.415367 (+ 1027us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.415367 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.415402 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.415433 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.415614 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.415619 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.415626 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.415692 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.415734 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260923 22:08:13.416244 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--builtin_ntp_servers=127.1.161.148:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 22:08:13.416349 6588 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41747
I20260923 22:08:13.416414 6588 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.416540 6588 heartbeater.cc:511] Master 127.1.161.190:41747 requested a full tablet report, sending...
I20260923 22:08:13.416834 6269 ts_manager.cc:194] Registered new tserver with Master: 33f05d74011b44e08b0363bea26b0b98 (127.1.161.130:41685)
W20260923 22:08:13.485384 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:13.485551 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:13.485574 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:13.486507 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:13.486536 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:13.486555 6591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:13.487725 6591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:13.487880 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:13.488068 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:13.492182 6597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:13.492178 6599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.492278 6591 server_base.cc:1034] running on GCE node
W20260923 22:08:13.492383 6596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:13.492565 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:13.492759 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:13.493955 6591 hybrid_clock.cc:648] HybridClock initialized: now 1790201293493941 us; error 23 us; skew 500 ppm
I20260923 22:08:13.495311 6591 webserver.cc:460] Webserver started at http://127.1.161.131:43525/ using document root <none> and password file <none>
I20260923 22:08:13.495512 6591 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:13.495584 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:13.495700 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:13.496614 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "cb977039b0fa41038aebcf953540ae0c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "9125709fe0db271fd55b8435d9fe0e52"
server_key_iv: "31c4f0fe2008feb9d1e7f91a5fc90413"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.496943 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "cb977039b0fa41038aebcf953540ae0c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "9125709fe0db271fd55b8435d9fe0e52"
server_key_iv: "31c4f0fe2008feb9d1e7f91a5fc90413"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.498018 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:13.498576 6605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.498720 6591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:13.498797 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "cb977039b0fa41038aebcf953540ae0c"
format_stamp: "Formatted at 2026-09-23 22:08:13 on dist-test-slave-npvd"
server_key: "9125709fe0db271fd55b8435d9fe0e52"
server_key_iv: "31c4f0fe2008feb9d1e7f91a5fc90413"
server_key_version: "encryptionkey@0"
I20260923 22:08:13.498868 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:13.501711 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:13.501920 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:13.502029 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:13.502197 6591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:13.502465 6591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:13.502511 6591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.502545 6591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:13.502570 6591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:13.507654 6591 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:41047
I20260923 22:08:13.507683 6718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:41047 every 8 connection(s)
I20260923 22:08:13.507929 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:13.510105 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6591
I20260923 22:08:13.510217 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:13.510314 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.508628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:39415 (local address 127.1.161.190:41747)
0923 22:08:13.508676 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.508678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.508687 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:13.509044 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.509045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.509051 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.509104 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.510057 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.510058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.510059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.510062 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.510076 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.510080 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.510144 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.510227 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.510242 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:13.510398 1670 external_mini_cluster.cc:1428] Setting key bb0f5ab5caf10d35ff71ae1ff3d42478
I20260923 22:08:13.510468 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.508437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41747 (local address 127.1.161.131:39415)
0923 22:08:13.508586 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.508616 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.508954 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.509156 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.509159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.509166 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.509914 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.509914 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.509935 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.509969 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.510145 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.510149 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.510152 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.510226 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.510278 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260923 22:08:13.510828 6719 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41747
I20260923 22:08:13.510910 6719 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:13.511031 6719 heartbeater.cc:511] Master 127.1.161.190:41747 requested a full tablet report, sending...
I20260923 22:08:13.511368 6268 ts_manager.cc:194] Registered new tserver with Master: cb977039b0fa41038aebcf953540ae0c (127.1.161.131:41047)
I20260923 22:08:13.521800 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:13.524114 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.523033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.1.161.190:41747)
0923 22:08:13.523084 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.523085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.523210 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:08:13.523293 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.523294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.523299 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.523356 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.523854 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.523855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.523856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.523860 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.523872 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.523876 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.523952 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.524012 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.524039 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:13.525549 6268 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:13.526933 6268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:13.530143 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.528905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.1.161.129:46687)
0923 22:08:13.529029 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.529033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.529044 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:13.529070 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.529072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.529081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.529399 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.529855 (+ 456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.529861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.529862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.529866 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.529897 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.529918 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.529987 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.530050 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.530066 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260923 22:08:13.530167 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.528742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41047 (local address 127.0.0.1:55398)
0923 22:08:13.529045 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.529052 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.529086 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.529232 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.529232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.529234 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.529817 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.529817 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.529821 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.529830 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.530024 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.530024 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.530025 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.530088 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.530099 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":2,"thread_start_us":65,"threads_started":1}
I20260923 22:08:13.530191 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.528808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.1.161.131:41047)
0923 22:08:13.529057 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.529060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.529091 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:13.529120 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.529122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.529128 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.529225 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.529897 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.529902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.529903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.529907 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.529937 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.529953 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.529985 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.530104 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.530119 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260923 22:08:13.530587 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.528742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46687 (local address 127.0.0.1:43250)
0923 22:08:13.528886 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.528914 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.528958 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.529186 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.529192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.529198 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.529702 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.529702 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.529715 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.529737 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.529978 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.529983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.529985 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.530519 (+ 534us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.530531 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":53,"threads_started":1}
I20260923 22:08:13.530656 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.529096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57374 (local address 127.1.161.130:41685)
0923 22:08:13.529529 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.529533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.529543 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:13.529569 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.529571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.529579 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.529673 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.530383 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.530390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.530390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.530394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.530421 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.530442 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.530489 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.530564 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.530576 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":62,"threads_started":1}
I20260923 22:08:13.530663 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.528924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:41685 (local address 127.0.0.1:57374)
0923 22:08:13.529179 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.529187 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.529215 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.529919 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.529920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.529923 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.530297 (+ 374us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.530298 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.530304 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.530317 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.530502 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.530502 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.530503 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.530557 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.530569 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":63,"threads_started":1}
I20260923 22:08:13.530730 6653 tablet_service.cc:1467] Processing CreateTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 (DEFAULT_TABLE table=test-table [id=fabd38d0b5374591a25d1e3df762f832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.531026 6653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf45710f8e8c40e5a9aa87600b79aef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.531193 6391 tablet_service.cc:1467] Processing CreateTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 (DEFAULT_TABLE table=test-table [id=fabd38d0b5374591a25d1e3df762f832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.531226 6522 tablet_service.cc:1467] Processing CreateTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 (DEFAULT_TABLE table=test-table [id=fabd38d0b5374591a25d1e3df762f832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:13.531525 6391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf45710f8e8c40e5a9aa87600b79aef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.531534 6522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf45710f8e8c40e5a9aa87600b79aef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:13.533794 6733 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Bootstrap starting.
I20260923 22:08:13.534176 6734 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Bootstrap starting.
I20260923 22:08:13.534545 6733 tablet_bootstrap.cc:654] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.534686 6735 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Bootstrap starting.
I20260923 22:08:13.534739 6734 tablet_bootstrap.cc:654] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.535012 6734 log.cc:824] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.535233 6733 log.cc:824] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.535272 6735 tablet_bootstrap.cc:654] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:13.535580 6735 log.cc:824] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:13.535722 6734 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: No bootstrap required, opened a new log
I20260923 22:08:13.535797 6734 ts_tablet_manager.cc:1397] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:08:13.536192 6735 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: No bootstrap required, opened a new log
I20260923 22:08:13.536263 6735 ts_tablet_manager.cc:1397] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:08:13.537273 6734 raft_consensus.cc:348] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.537397 6734 raft_consensus.cc:374] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.537418 6734 raft_consensus.cc:729] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb977039b0fa41038aebcf953540ae0c, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.537513 6734 consensus_queue.cc:260] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.537737 6734 ts_tablet_manager.cc:1428] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:08:13.537773 6719 heartbeater.cc:503] Master 127.1.161.190:41747 was elected leader, sending a full tablet report...
I20260923 22:08:13.538066 6733 tablet_bootstrap.cc:492] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: No bootstrap required, opened a new log
I20260923 22:08:13.538126 6733 ts_tablet_manager.cc:1397] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260923 22:08:13.538326 6735 raft_consensus.cc:348] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.538477 6735 raft_consensus.cc:374] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.538523 6735 raft_consensus.cc:729] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436c6530c8b84590aa11581f102ca88c, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.538622 6735 consensus_queue.cc:260] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.538893 6457 heartbeater.cc:503] Master 127.1.161.190:41747 was elected leader, sending a full tablet report...
I20260923 22:08:13.538875 6735 ts_tablet_manager.cc:1428] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:08:13.540472 6733 raft_consensus.cc:348] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.540608 6733 raft_consensus.cc:374] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:13.540663 6733 raft_consensus.cc:729] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f05d74011b44e08b0363bea26b0b98, State: Initialized, Role: FOLLOWER
I20260923 22:08:13.540742 6733 consensus_queue.cc:260] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.540905 6733 ts_tablet_manager.cc:1428] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 22:08:13.540973 6588 heartbeater.cc:503] Master 127.1.161.190:41747 was elected leader, sending a full tablet report...
W20260923 22:08:13.558173 6458 tablet.cc:2369] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:13.663463 6589 tablet.cc:2369] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:13.758392 6720 tablet.cc:2369] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:13.889418 6740 raft_consensus.cc:482] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:13.889531 6740 raft_consensus.cc:504] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.890002 6740 leader_election.cc:288] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb977039b0fa41038aebcf953540ae0c (127.1.161.131:41047), 33f05d74011b44e08b0363bea26b0b98 (127.1.161.130:41685)
I20260923 22:08:13.891113 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.890245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:46327 (local address 127.1.161.130:41685)
0923 22:08:13.890297 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.890299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.890308 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:13.890381 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.890382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.890388 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.890432 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.890922 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.890924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.890924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.890928 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.890941 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.890945 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.890997 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.891042 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.891053 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:13.891144 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.890138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:41685 (local address 127.1.161.129:46327)
0923 22:08:13.890250 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.890260 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.890303 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.890444 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.890445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.890447 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.890846 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.890846 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.890852 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.890861 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.890993 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.890994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.890995 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.891041 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.891053 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260923 22:08:13.891429 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.890232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:34567 (local address 127.1.161.131:41047)
0923 22:08:13.890275 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.890277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.890314 (+ 37us) server_negotiation.cc:407] Connection header received
0923 22:08:13.890422 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.890423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.890428 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.890479 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.891117 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.891118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.891118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.891124 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.891139 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.891143 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.891219 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.891318 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.891334 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:13.891429 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf45710f8e8c40e5a9aa87600b79aef6" candidate_uuid: "436c6530c8b84590aa11581f102ca88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f05d74011b44e08b0363bea26b0b98" is_pre_election: true
I20260923 22:08:13.891594 6542 raft_consensus.cc:2454] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 436c6530c8b84590aa11581f102ca88c in term 0.
I20260923 22:08:13.891629 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.890138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41047 (local address 127.1.161.129:34567)
0923 22:08:13.890251 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.890263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:13.890327 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:13.890482 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:13.890483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:13.890498 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:13.891013 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:13.891014 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:13.891020 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:13.891037 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:13.891244 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:13.891244 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:13.891245 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:13.891545 (+ 300us) client_negotiation.cc:240] Negotiation successful
0923 22:08:13.891560 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":50,"threads_started":1}
I20260923 22:08:13.891875 6344 leader_election.cc:302] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33f05d74011b44e08b0363bea26b0b98, 436c6530c8b84590aa11581f102ca88c; no voters:
I20260923 22:08:13.892105 6673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf45710f8e8c40e5a9aa87600b79aef6" candidate_uuid: "436c6530c8b84590aa11581f102ca88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb977039b0fa41038aebcf953540ae0c" is_pre_election: true
I20260923 22:08:13.892248 6673 raft_consensus.cc:2454] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 436c6530c8b84590aa11581f102ca88c in term 0.
I20260923 22:08:13.892365 6740 raft_consensus.cc:2781] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:13.892424 6740 raft_consensus.cc:482] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:13.892449 6740 raft_consensus.cc:3037] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.893453 6740 raft_consensus.cc:504] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.893676 6740 leader_election.cc:288] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [CANDIDATE]: Term 1 election: Requested vote from peers cb977039b0fa41038aebcf953540ae0c (127.1.161.131:41047), 33f05d74011b44e08b0363bea26b0b98 (127.1.161.130:41685)
I20260923 22:08:13.893754 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf45710f8e8c40e5a9aa87600b79aef6" candidate_uuid: "436c6530c8b84590aa11581f102ca88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f05d74011b44e08b0363bea26b0b98"
I20260923 22:08:13.893785 6673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf45710f8e8c40e5a9aa87600b79aef6" candidate_uuid: "436c6530c8b84590aa11581f102ca88c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb977039b0fa41038aebcf953540ae0c"
I20260923 22:08:13.893843 6542 raft_consensus.cc:3037] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.893869 6673 raft_consensus.cc:3037] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:13.894780 6542 raft_consensus.cc:2454] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 436c6530c8b84590aa11581f102ca88c in term 1.
I20260923 22:08:13.894826 6673 raft_consensus.cc:2454] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 436c6530c8b84590aa11581f102ca88c in term 1.
I20260923 22:08:13.894974 6345 leader_election.cc:302] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436c6530c8b84590aa11581f102ca88c, cb977039b0fa41038aebcf953540ae0c; no voters:
I20260923 22:08:13.895049 6740 raft_consensus.cc:2781] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:13.895198 6740 raft_consensus.cc:686] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 1 LEADER]: Becoming Leader. State: Replica: 436c6530c8b84590aa11581f102ca88c, State: Running, Role: LEADER
I20260923 22:08:13.895288 6740 consensus_queue.cc:237] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } }
I20260923 22:08:13.896045 6268 catalog_manager.cc:5495] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c reported cstate change: term changed from 0 to 1, leader changed from <none> to 436c6530c8b84590aa11581f102ca88c (127.1.161.129). New cstate: current_term: 1 leader_uuid: "436c6530c8b84590aa11581f102ca88c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "436c6530c8b84590aa11581f102ca88c" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:13.970570 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.969636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.1.161.129:46687)
0923 22:08:13.969676 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:13.969677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:13.969685 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:13.969752 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:13.969753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:13.969762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:13.969800 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:13.970341 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:13.970342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:13.970342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:13.970345 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:13.970355 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:13.970358 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:13.970392 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:13.970464 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:08:13.970477 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:13.972226 6542 raft_consensus.cc:1261] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 1 FOLLOWER]: Refusing update from remote peer 436c6530c8b84590aa11581f102ca88c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.972226 6673 raft_consensus.cc:1261] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 1 FOLLOWER]: Refusing update from remote peer 436c6530c8b84590aa11581f102ca88c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:13.972566 6744 consensus_queue.cc:1035] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33f05d74011b44e08b0363bea26b0b98" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.972692 6740 consensus_queue.cc:1035] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb977039b0fa41038aebcf953540ae0c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:13.975210 6748 mvcc.cc:205] Tried to move back new op lower bound from 7332664500107857920 to 7332664499795824640. Current Snapshot: MvccSnapshot[applied={T|T < 7332664500107857920}]
I20260923 22:08:13.976935 6269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.977025 6269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:13.977720 6269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092f7609c23f486986bdf9832dd637c6: Sending DeleteTablet for 3 replicas of tablet bf45710f8e8c40e5a9aa87600b79aef6
I20260923 22:08:13.978019 6653 tablet_service.cc:1514] Processing DeleteTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='slave'} at 127.0.0.1:55398
I20260923 22:08:13.978041 6391 tablet_service.cc:1514] Processing DeleteTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='slave'} at 127.0.0.1:43250
I20260923 22:08:13.978049 6522 tablet_service.cc:1514] Processing DeleteTablet for tablet bf45710f8e8c40e5a9aa87600b79aef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:13 UTC) from {username='slave'} at 127.0.0.1:57374
I20260923 22:08:13.978230 6757 tablet_replica.cc:330] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: stopping tablet replica
I20260923 22:08:13.978387 6757 raft_consensus.cc:2229] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.978484 6757 raft_consensus.cc:2258] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.978597 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6329
I20260923 22:08:13.978696 6758 tablet_replica.cc:330] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c: stopping tablet replica
I20260923 22:08:13.978772 6758 raft_consensus.cc:2229] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:13.978870 6757 ts_tablet_manager.cc:1905] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.978874 6758 raft_consensus.cc:2258] T bf45710f8e8c40e5a9aa87600b79aef6 P 436c6530c8b84590aa11581f102ca88c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.978310 6756 tablet_replica.cc:330] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: stopping tablet replica
I20260923 22:08:13.980284 6756 raft_consensus.cc:2229] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:13.980398 6756 raft_consensus.cc:2258] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:13.980424 6757 ts_tablet_manager.cc:1918] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.980520 6757 log.cc:1196] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/bf45710f8e8c40e5a9aa87600b79aef6
I20260923 22:08:13.980747 6757 ts_tablet_manager.cc:1939] T bf45710f8e8c40e5a9aa87600b79aef6 P cb977039b0fa41038aebcf953540ae0c: Deleting consensus metadata
I20260923 22:08:13.980820 6756 ts_tablet_manager.cc:1905] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:13.981103 6254 catalog_manager.cc:4841] TS cb977039b0fa41038aebcf953540ae0c (127.1.161.131:41047): tablet bf45710f8e8c40e5a9aa87600b79aef6 (table test-table [id=fabd38d0b5374591a25d1e3df762f832]) successfully deleted
I20260923 22:08:13.982688 6756 ts_tablet_manager.cc:1918] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:13.982774 6756 log.cc:1196] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/bf45710f8e8c40e5a9aa87600b79aef6
I20260923 22:08:13.983053 6756 ts_tablet_manager.cc:1939] T bf45710f8e8c40e5a9aa87600b79aef6 P 33f05d74011b44e08b0363bea26b0b98: Deleting consensus metadata
I20260923 22:08:13.983458 6253 catalog_manager.cc:4841] TS 33f05d74011b44e08b0363bea26b0b98 (127.1.161.130:41685): tablet bf45710f8e8c40e5a9aa87600b79aef6 (table test-table [id=fabd38d0b5374591a25d1e3df762f832]) successfully deleted
W20260923 22:08:13.985105 6256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:46687 (error 108)
I20260923 22:08:13.985548 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:13.985446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46687 (local address 127.0.0.1:43268)
0923 22:08:13.985493 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:13.985525 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:08:13.985651 6256 catalog_manager.cc:4584] TS 436c6530c8b84590aa11581f102ca88c (127.1.161.129:46687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf45710f8e8c40e5a9aa87600b79aef6: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46687: connect: Connection refused (error 111)
I20260923 22:08:13.985769 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6460
I20260923 22:08:13.990244 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6591
I20260923 22:08:13.994887 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6238
2026-09-23T22:08:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (922 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 22:08:14.013374 1670 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:14.014710 1670 webserver.cc:460] Webserver started at https://127.0.0.1:45123/ using document root <none> and password file <none>
I20260923 22:08:14.014822 1670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:14.015015 1670 webserver.cc:460] Webserver started at http://127.0.0.1:42545/ using document root <none> and password file <none>
2026-09-23T22:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:14Z Disabled control of system clock
I20260923 22:08:14.021364 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38829
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38829
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45123/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/testchainca.cert with env {}
W20260923 22:08:14.090840 6765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:14.091018 6765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:14.091050 6765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:14.092221 6765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:14.092278 6765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:14.092299 6765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:14.092320 6765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:14.092346 6765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:14.092372 6765 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:14.092396 6765 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45123/jwks.json
I20260923 22:08:14.093878 6765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45123/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:14.094149 6765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:14.094332 6765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:14.098233 6771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.098260 6770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.098276 6773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:14.098543 6765 server_base.cc:1034] running on GCE node
I20260923 22:08:14.098766 6765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:14.098994 6765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:14.100136 6765 hybrid_clock.cc:648] HybridClock initialized: now 1790201294100126 us; error 21 us; skew 500 ppm
I20260923 22:08:14.101162 6765 webserver.cc:460] Webserver started at http://127.1.161.190:34709/ using document root <none> and password file <none>
I20260923 22:08:14.101364 6765 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:14.101420 6765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:14.101526 6765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:14.102315 6765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "90d00b7dd4a04b7f8aed91030803ac93"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b8d8a91da7dae4a43a74bb7066ebfd64"
server_key_iv: "3e87e404128098cc2a611685e40c3695"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.102604 6765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "90d00b7dd4a04b7f8aed91030803ac93"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b8d8a91da7dae4a43a74bb7066ebfd64"
server_key_iv: "3e87e404128098cc2a611685e40c3695"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.103840 6765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:14.104521 6779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.104652 6765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:14.104732 6765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "90d00b7dd4a04b7f8aed91030803ac93"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b8d8a91da7dae4a43a74bb7066ebfd64"
server_key_iv: "3e87e404128098cc2a611685e40c3695"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.104794 6765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:14.121176 6765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:14.121403 6765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:14.121523 6765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:14.124931 6765 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38829
I20260923 22:08:14.124950 6831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38829 every 8 connection(s)
I20260923 22:08:14.125204 6765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:14.126044 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.127813 6832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93: Bootstrap starting.
I20260923 22:08:14.128281 6832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.128527 6832 log.cc:824] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:14.129063 6832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93: No bootstrap required, opened a new log
I20260923 22:08:14.130292 6832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } }
I20260923 22:08:14.130395 6832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.130443 6832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d00b7dd4a04b7f8aed91030803ac93, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.130554 6832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } }
I20260923 22:08:14.130604 6832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:14.130620 6832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:14.130676 6832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.131083 6832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } }
I20260923 22:08:14.131171 6832 leader_election.cc:302] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d00b7dd4a04b7f8aed91030803ac93; no voters:
I20260923 22:08:14.131338 6832 leader_election.cc:288] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:14.131382 6835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.131523 6835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [term 1 LEADER]: Becoming Leader. State: Replica: 90d00b7dd4a04b7f8aed91030803ac93, State: Running, Role: LEADER
I20260923 22:08:14.131554 6832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:14.131603 6835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } }
I20260923 22:08:14.131902 6836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90d00b7dd4a04b7f8aed91030803ac93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } } }
I20260923 22:08:14.131983 6836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.132072 6837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d00b7dd4a04b7f8aed91030803ac93. Latest consensus state: current_term: 1 leader_uuid: "90d00b7dd4a04b7f8aed91030803ac93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d00b7dd4a04b7f8aed91030803ac93" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38829 } } }
I20260923 22:08:14.132174 6837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.132408 6842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:14.132835 6842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:14.134181 6842 catalog_manager.cc:1347] Generated new cluster ID: d8e5a0fd1c0d4cbaa2ea006c893c9d7b
I20260923 22:08:14.134249 6842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:14.134732 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6765
I20260923 22:08:14.134819 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:14.134979 1670 external_mini_cluster.cc:1428] Setting key 92f283378df0ce8e105e915a4cc1d74e
I20260923 22:08:14.139923 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.135321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.1.161.190:38829)
0923 22:08:14.135476 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.135483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.135644 (+ 161us) server_negotiation.cc:407] Connection header received
0923 22:08:14.135751 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.135757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.135817 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.135934 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.136114 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.137239 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.138186 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.138684 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.138772 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.139378 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:14.139382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:14.139382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:14.139539 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:14.139570 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:14.139581 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:14.139632 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.139712 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.139771 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260923 22:08:14.140672 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:14.142192 6842 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:14.142573 6842 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:14.149824 6842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90d00b7dd4a04b7f8aed91030803ac93: Generated new TSK 0
I20260923 22:08:14.149950 6842 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:08:14.167117 6862 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 22:08:14.167207 6862 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:08:14.167392 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.154735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.1.161.190:38829)
0923 22:08:14.154763 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.154765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.154884 (+ 119us) server_negotiation.cc:407] Connection header received
0923 22:08:14.154954 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.154955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.154962 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.155009 (+ 47us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:14.155108 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.155546 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.156461 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.156604 (+ 143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.156644 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.156807 (+ 163us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:14.167242 (+ 10435us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:08:14.167358 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":13}
W20260923 22:08:14.167439 6855 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 22:08:14.167663 6861 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:14.154693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38829 (local address 127.0.0.1:58800)
0923 22:08:14.154830 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.154839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.154891 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.155017 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.155018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.155019 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:14.155060 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.155061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.155607 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.155608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.156395 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.156397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.156661 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.156662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.156709 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.156729 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:14.167340 (+ 10611us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 22:08:14.167426 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.161.190:38829: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.161.190:38829: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45123/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 22:08:14.168020 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6765
2026-09-23T22:08:14Z chronyd exiting
I20260923 22:08:14.202982 1670 test_util.cc:175] -----------------------------------------------
I20260923 22:08:14.203047 1670 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790201274647240-1670-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (190 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 22:08:14.203733 1670 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:14.204294 1670 webserver.cc:460] Webserver started at https://127.0.0.1:40625/ using document root <none> and password file <none>
I20260923 22:08:14.204325 1670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:14.204435 1670 webserver.cc:460] Webserver started at http://127.0.0.1:38195/ using document root <none> and password file <none>
2026-09-23T22:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:14Z Disabled control of system clock
I20260923 22:08:14.210475 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40713
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40713
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/testchainca.cert with env {}
W20260923 22:08:14.278911 6868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:14.279069 6868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:14.279088 6868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:14.280066 6868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:14.280099 6868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:14.280110 6868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:14.280121 6868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:14.280131 6868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:14.280141 6868 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:14.280150 6868 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40625/jwks.json
I20260923 22:08:14.281316 6868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:14.281461 6868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:14.281594 6868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:14.285648 6876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.285640 6873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.285652 6874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:14.285785 6868 server_base.cc:1034] running on GCE node
I20260923 22:08:14.285972 6868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:14.286204 6868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:14.287336 6868 hybrid_clock.cc:648] HybridClock initialized: now 1790201294287320 us; error 30 us; skew 500 ppm
I20260923 22:08:14.288357 6868 webserver.cc:460] Webserver started at http://127.1.161.190:38973/ using document root <none> and password file <none>
I20260923 22:08:14.288568 6868 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:14.288620 6868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:14.288727 6868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:14.289546 6868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "fec8697449fc452e8bd68edf49d6981d"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "effb34269b6b48876a45307340bc0e36"
server_key_iv: "dc85e46b105a7f237b43ebe1bbd820fd"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.289844 6868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "fec8697449fc452e8bd68edf49d6981d"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "effb34269b6b48876a45307340bc0e36"
server_key_iv: "dc85e46b105a7f237b43ebe1bbd820fd"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.290901 6868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:14.291532 6882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.291687 6868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.291764 6868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "fec8697449fc452e8bd68edf49d6981d"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "effb34269b6b48876a45307340bc0e36"
server_key_iv: "dc85e46b105a7f237b43ebe1bbd820fd"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.291831 6868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:14.299504 6868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:14.299717 6868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:14.299829 6868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:14.302960 6868 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40713
I20260923 22:08:14.302994 6934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40713 every 8 connection(s)
I20260923 22:08:14.303346 6868 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:14.303581 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6868
I20260923 22:08:14.303694 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:14.303884 1670 external_mini_cluster.cc:1428] Setting key c5d11e0cb14162ad406f1a596a96241c
I20260923 22:08:14.304471 6935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.307345 6935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d: Bootstrap starting.
I20260923 22:08:14.307957 6935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.308234 6935 log.cc:824] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:14.308969 6935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d: No bootstrap required, opened a new log
I20260923 22:08:14.309346 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.304340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.1.161.190:40713)
0923 22:08:14.304553 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.304561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.304588 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:14.305079 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.305088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.305163 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.305314 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.305474 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.306710 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.307419 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.307941 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.308062 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.308657 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:14.308662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:14.308662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:14.308847 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:14.308877 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:14.308887 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:14.308944 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.309067 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.309146 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260923 22:08:14.310256 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:14.310741 6935 raft_consensus.cc:348] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } }
I20260923 22:08:14.310875 6935 raft_consensus.cc:374] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.310933 6935 raft_consensus.cc:729] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec8697449fc452e8bd68edf49d6981d, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.311044 6935 consensus_queue.cc:260] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } }
I20260923 22:08:14.311105 6935 raft_consensus.cc:388] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:14.311151 6935 raft_consensus.cc:482] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:14.311218 6935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.311646 6935 raft_consensus.cc:504] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } }
I20260923 22:08:14.311717 6935 leader_election.cc:302] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fec8697449fc452e8bd68edf49d6981d; no voters:
I20260923 22:08:14.311810 6935 leader_election.cc:288] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:14.311869 6940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.312004 6940 raft_consensus.cc:686] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [term 1 LEADER]: Becoming Leader. State: Replica: fec8697449fc452e8bd68edf49d6981d, State: Running, Role: LEADER
I20260923 22:08:14.312018 6935 sys_catalog.cc:564] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:14.312084 6940 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } }
I20260923 22:08:14.312412 6941 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fec8697449fc452e8bd68edf49d6981d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } } }
I20260923 22:08:14.312425 6942 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec8697449fc452e8bd68edf49d6981d. Latest consensus state: current_term: 1 leader_uuid: "fec8697449fc452e8bd68edf49d6981d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec8697449fc452e8bd68edf49d6981d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40713 } } }
I20260923 22:08:14.312556 6941 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.312575 6942 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.313124 6952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:14.313623 6952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:14.314857 6952 catalog_manager.cc:1347] Generated new cluster ID: 44bcf29880714bc0ad0e5ff27f6a7215
I20260923 22:08:14.314908 6952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:14.336126 6952 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:14.336624 6952 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:14.342238 6952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fec8697449fc452e8bd68edf49d6981d: Generated new TSK 0
I20260923 22:08:14.342396 6952 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 22:08:14.364703 6965 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 22:08:14.364771 6965 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 22:08:14.364990 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.352307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.1.161.190:40713)
0923 22:08:14.352352 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.352354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.352428 (+ 74us) server_negotiation.cc:407] Connection header received
0923 22:08:14.352469 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.352470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.352480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.352550 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:14.352681 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.353372 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.354172 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.354505 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.354573 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.354674 (+ 101us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:14.364829 (+ 10155us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:08:14.364939 (+ 110us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260923 22:08:14.365033 6938 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 22:08:14.365129 6964 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:14.352239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40713 (local address 127.0.0.1:39920)
0923 22:08:14.352366 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.352375 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.352427 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.352574 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.352575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.352577 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:14.352621 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.352622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.353504 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.353506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.354092 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.354093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.354573 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.354574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.354608 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.354612 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:14.364921 (+ 10309us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 22:08:14.365009 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.161.190:40713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260923 22:08:14.365432 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6868
2026-09-23T22:08:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (197 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 22:08:14.401564 1670 mini_oidc.cc:152] Starting JWKS server
I20260923 22:08:14.402103 1670 webserver.cc:460] Webserver started at https://127.0.0.1:44199/ using document root <none> and password file <none>
I20260923 22:08:14.402158 1670 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 22:08:14.402308 1670 webserver.cc:460] Webserver started at http://127.0.0.1:33919/ using document root <none> and password file <none>
2026-09-23T22:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:14Z Disabled control of system clock
I20260923 22:08:14.408730 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33387
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33387
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44199/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:08:14.479120 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:14.479341 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:14.479375 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:14.480796 6971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:14.480845 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:14.480863 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:14.480885 6971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:14.480911 6971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:14.480934 6971 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:14.480957 6971 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44199/jwks.json
I20260923 22:08:14.482088 6971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33387
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44199/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:14.482307 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:14.482556 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:14.486744 6976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.486740 6977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.486742 6979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:14.486904 6971 server_base.cc:1034] running on GCE node
I20260923 22:08:14.487107 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:14.487351 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:14.488520 6971 hybrid_clock.cc:648] HybridClock initialized: now 1790201294488496 us; error 40 us; skew 500 ppm
I20260923 22:08:14.489671 6971 webserver.cc:460] Webserver started at http://127.1.161.190:44529/ using document root <none> and password file <none>
I20260923 22:08:14.489869 6971 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:14.489928 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:14.490053 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:14.490886 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "77fe447a1a6546f7a34865811a664d5b"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b68b9665d6098f0ec25be2700ec76f78"
server_key_iv: "53b7c073bce3eed6cee13d54b5d84461"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.491226 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "77fe447a1a6546f7a34865811a664d5b"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b68b9665d6098f0ec25be2700ec76f78"
server_key_iv: "53b7c073bce3eed6cee13d54b5d84461"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.492254 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:14.492863 6985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.493007 6971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:14.493085 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "77fe447a1a6546f7a34865811a664d5b"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "b68b9665d6098f0ec25be2700ec76f78"
server_key_iv: "53b7c073bce3eed6cee13d54b5d84461"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.493175 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:14.506497 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:14.506726 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:14.506843 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:14.510141 6971 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:33387
I20260923 22:08:14.510166 7037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:33387 every 8 connection(s)
I20260923 22:08:14.510421 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:14.510912 7038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.511824 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 6971
I20260923 22:08:14.511916 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:14.512074 1670 external_mini_cluster.cc:1428] Setting key 9ca1bc4ffc23a524e871c85a24ed4552
I20260923 22:08:14.513247 7038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b: Bootstrap starting.
I20260923 22:08:14.513768 7038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.514043 7038 log.cc:824] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:14.514647 7038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b: No bootstrap required, opened a new log
I20260923 22:08:14.516198 7038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } }
I20260923 22:08:14.516326 7038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.516368 7038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77fe447a1a6546f7a34865811a664d5b, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.516486 7038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } }
I20260923 22:08:14.516566 7038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:14.516597 7038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:14.516656 7038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.516839 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.512462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.1.161.190:33387)
0923 22:08:14.512647 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.512658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.512677 (+ 19us) server_negotiation.cc:407] Connection header received
0923 22:08:14.513065 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.513071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.513135 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.513278 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.513461 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.514400 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.515090 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.515497 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.515608 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.516150 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:14.516154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:14.516154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:14.516295 (+ 141us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:14.516323 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:14.516332 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:14.516396 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.516516 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.516688 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260923 22:08:14.517179 7038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } }
I20260923 22:08:14.517274 7038 leader_election.cc:302] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77fe447a1a6546f7a34865811a664d5b; no voters:
I20260923 22:08:14.517416 7038 leader_election.cc:288] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:14.517436 7043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.517503 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:14.517604 7043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [term 1 LEADER]: Becoming Leader. State: Replica: 77fe447a1a6546f7a34865811a664d5b, State: Running, Role: LEADER
I20260923 22:08:14.517632 7038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:14.517685 7043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } }
I20260923 22:08:14.517961 7043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77fe447a1a6546f7a34865811a664d5b. Latest consensus state: current_term: 1 leader_uuid: "77fe447a1a6546f7a34865811a664d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } } }
I20260923 22:08:14.518046 7043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.517967 7044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77fe447a1a6546f7a34865811a664d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77fe447a1a6546f7a34865811a664d5b" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33387 } } }
I20260923 22:08:14.518131 7044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.518276 7051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:14.518716 7051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:14.519909 7051 catalog_manager.cc:1347] Generated new cluster ID: 3a3ff36629644828afd43d103c0019db
I20260923 22:08:14.519955 7051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:14.537793 7051 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:14.538219 7051 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:14.540860 7051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77fe447a1a6546f7a34865811a664d5b: Generated new TSK 0
I20260923 22:08:14.540999 7051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:14.548887 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.545971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.1.161.190:33387)
0923 22:08:14.546005 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.546006 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.546086 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:08:14.546148 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.546149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.546164 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.546212 (+ 48us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:14.546325 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.546731 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.547659 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.547812 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.547855 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.548012 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:14.548732 (+ 720us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44199/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:08:14.548836 (+ 104us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44199/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:08:14.548929 7041 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44199/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 22:08:14.549202 7067 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:14.545870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33387 (local address 127.0.0.1:39430)
0923 22:08:14.546021 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.546031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.546092 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.546213 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.546214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.546216 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:14.546276 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.546277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.546801 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.546801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.547589 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.547591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.547874 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.547875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.547929 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.547934 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:14.548825 (+ 891us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44199/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 22:08:14.548919 (+ 94us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.161.190:33387: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44199/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260923 22:08:14.549407 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 6971
E20260923 22:08:14.558609 7068 webserver.cc:533] Webserver: sslize failed:
E20260923 22:08:14.558669 7068 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T22:08:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (184 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T22:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:14Z Disabled control of system clock
I20260923 22:08:14.592358 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40441
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40441
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/testchainca.cert with env {}
W20260923 22:08:14.661145 7072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:14.661514 7072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:14.661604 7072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:14.662752 7072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:14.662813 7072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:14.662835 7072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:14.662865 7072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:14.662892 7072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 22:08:14.662923 7072 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 22:08:14.662948 7072 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 22:08:14.664934 7072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:14.665174 7072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:14.665360 7072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:14.669494 7077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:14.669561 7072 server_base.cc:1034] running on GCE node
W20260923 22:08:14.669425 7080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:14.669466 7078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:14.669884 7072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:14.670137 7072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:14.671291 7072 hybrid_clock.cc:648] HybridClock initialized: now 1790201294671281 us; error 27 us; skew 500 ppm
I20260923 22:08:14.672304 7072 webserver.cc:460] Webserver started at http://127.1.161.190:34289/ using document root <none> and password file <none>
I20260923 22:08:14.672515 7072 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:14.672591 7072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:14.672716 7072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:14.673679 7072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "34fcf254c4d44fc9b4a65dc787986eb8"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "c24a43c34b663a9f9ac82379ac50aac2"
server_key_iv: "c1fd89cb48b5d36da836ba40a32b2382"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.674010 7072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "34fcf254c4d44fc9b4a65dc787986eb8"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "c24a43c34b663a9f9ac82379ac50aac2"
server_key_iv: "c1fd89cb48b5d36da836ba40a32b2382"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.675141 7072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:14.675788 7086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:14.675921 7072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:14.675999 7072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "34fcf254c4d44fc9b4a65dc787986eb8"
format_stamp: "Formatted at 2026-09-23 22:08:14 on dist-test-slave-npvd"
server_key: "c24a43c34b663a9f9ac82379ac50aac2"
server_key_iv: "c1fd89cb48b5d36da836ba40a32b2382"
server_key_version: "encryptionkey@0"
I20260923 22:08:14.676069 7072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:14.683838 7072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:14.684067 7072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:14.684197 7072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:14.687484 7072 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40441
I20260923 22:08:14.687508 7138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40441 every 8 connection(s)
I20260923 22:08:14.687762 7072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:14.688445 7139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:14.690097 7139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8: Bootstrap starting.
I20260923 22:08:14.690528 7139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:14.690734 7139 log.cc:824] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:14.691306 7139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8: No bootstrap required, opened a new log
I20260923 22:08:14.692620 7139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } }
I20260923 22:08:14.692724 7139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:14.692741 7139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fcf254c4d44fc9b4a65dc787986eb8, State: Initialized, Role: FOLLOWER
I20260923 22:08:14.692844 7139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } }
I20260923 22:08:14.692909 7139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:14.692929 7139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:14.692955 7139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:14.693394 7139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } }
I20260923 22:08:14.693470 7139 leader_election.cc:302] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34fcf254c4d44fc9b4a65dc787986eb8; no voters:
I20260923 22:08:14.693573 7139 leader_election.cc:288] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:14.693634 7142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:14.693799 7142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [term 1 LEADER]: Becoming Leader. State: Replica: 34fcf254c4d44fc9b4a65dc787986eb8, State: Running, Role: LEADER
I20260923 22:08:14.693830 7139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:14.693893 7142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } }
I20260923 22:08:14.694281 7143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } } }
I20260923 22:08:14.694432 7143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.694418 7144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34fcf254c4d44fc9b4a65dc787986eb8. Latest consensus state: current_term: 1 leader_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34fcf254c4d44fc9b4a65dc787986eb8" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40441 } } }
I20260923 22:08:14.694551 7144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:14.694834 7151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:14.695291 7151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:14.695731 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7072
I20260923 22:08:14.695832 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:14.695979 1670 external_mini_cluster.cc:1428] Setting key e86069e9614c10b5b0e20953867a80e8
I20260923 22:08:14.696888 7151 catalog_manager.cc:1347] Generated new cluster ID: 89a9f56e94724f77862941133e5a8e8a
I20260923 22:08:14.696944 7151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:14.701398 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.696437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.1.161.190:40441)
0923 22:08:14.696614 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.696622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.696653 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:14.696751 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.696756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.696815 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.697260 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:14.697349 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.698589 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.699537 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.700070 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.700189 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.700752 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:14.700756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:14.700756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:14.700969 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:14.701014 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:14.701028 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:14.701101 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:14.701173 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:08:14.701237 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260923 22:08:14.702159 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:08:14.703138 7151 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:14.703645 7151 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:14.710354 7151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34fcf254c4d44fc9b4a65dc787986eb8: Generated new TSK 0
I20260923 22:08:14.710496 7151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:14.790376 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:14.715643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.1.161.190:40441)
0923 22:08:14.715677 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:14.715679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:14.715687 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:14.715744 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:14.715745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:14.715756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:14.715851 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0923 22:08:14.715989 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.716464 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.717230 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.717413 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.717477 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.717655 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 22:08:14.790098 (+ 72443us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:08:14.790289 (+ 191us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33804: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:08:14.790448 7162 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33804: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 22:08:14.790468 7168 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:14.715482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40441 (local address 127.0.0.1:33804)
0923 22:08:14.715617 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:14.715627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:14.715679 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:14.715850 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:14.715851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:14.715853 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 22:08:14.715903 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.715905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.716566 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.716568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.717142 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:14.717144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:14.717495 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:14.717496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:14.717560 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:14.717565 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 22:08:14.790248 (+ 72683us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 22:08:14.790359 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.161.190:40441: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260923 22:08:14.790848 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7072
2026-09-23T22:08:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (221 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (74 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (73 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:14 dist-test-slave-npvd krb5kdc[7180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:14 dist-test-slave-npvd krb5kdc[7180](info): set up 2 sockets
Sep 23 22:08:14 dist-test-slave-npvd krb5kdc[7180](info): commencing operation
krb5kdc: starting...
W20260923 22:08:15.846946 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.891s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:15 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:15.945976 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46757
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:46757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:16.018018 7193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:16.018186 7193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:16.018205 7193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:16.019657 7193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:16.019703 7193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:16.019719 7193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:16.019737 7193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:16.019770 7193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:16.021585 7193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:46757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:16.021816 7193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:16.022004 7193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:16.026449 7201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:16.026454 7199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:16.026589 7193 server_base.cc:1034] running on GCE node
W20260923 22:08:16.026479 7198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:16.026865 7193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:16.027124 7193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:16.028322 7193 hybrid_clock.cc:648] HybridClock initialized: now 1790201296028305 us; error 31 us; skew 500 ppm
Sep 23 22:08:16 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:16.034157 7193 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:16.034533 7193 webserver.cc:460] Webserver started at http://127.1.161.190:37039/ using document root <none> and password file <none>
I20260923 22:08:16.034752 7193 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:16.034813 7193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:16.034932 7193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:16.035799 7193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "b2c2e7f371764beaae77f018ae14f5ec"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-npvd"
server_key: "a6956c1fc12355e710957ff4d22d9aa5"
server_key_iv: "119d8253def3edc3ffb687028237fc65"
server_key_version: "encryptionkey@0"
I20260923 22:08:16.036113 7193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "b2c2e7f371764beaae77f018ae14f5ec"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-npvd"
server_key: "a6956c1fc12355e710957ff4d22d9aa5"
server_key_iv: "119d8253def3edc3ffb687028237fc65"
server_key_version: "encryptionkey@0"
I20260923 22:08:16.037179 7193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:16.037731 7208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:16.037864 7193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:16.037942 7193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "b2c2e7f371764beaae77f018ae14f5ec"
format_stamp: "Formatted at 2026-09-23 22:08:16 on dist-test-slave-npvd"
server_key: "a6956c1fc12355e710957ff4d22d9aa5"
server_key_iv: "119d8253def3edc3ffb687028237fc65"
server_key_version: "encryptionkey@0"
I20260923 22:08:16.038008 7193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:16.047128 7193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:16.048734 7193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:16.048875 7193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:16.052577 7193 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:46757
I20260923 22:08:16.052609 7260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:46757 every 8 connection(s)
I20260923 22:08:16.052932 7193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:16.053717 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:16.055891 7261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec: Bootstrap starting.
I20260923 22:08:16.056450 7261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:16.056725 7261 log.cc:824] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:16.057320 7261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec: No bootstrap required, opened a new log
I20260923 22:08:16.058741 7261 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } }
I20260923 22:08:16.058877 7261 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:16.058920 7261 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c2e7f371764beaae77f018ae14f5ec, State: Initialized, Role: FOLLOWER
I20260923 22:08:16.059068 7261 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } }
I20260923 22:08:16.059126 7261 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:16.059167 7261 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:16.059242 7261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:16.059476 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7193
I20260923 22:08:16.059578 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:16.059779 1670 external_mini_cluster.cc:1428] Setting key 8cbf4635eb097fcd3abf55def807b08f
Sep 23 22:08:16 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:16.068504 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:16.060171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.1.161.190:46757)
0923 22:08:16.060353 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:16.060363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:16.060392 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:08:16.060482 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:16.060486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:16.060527 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:16.060641 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:16.061199 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:16.062181 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:16.062903 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:16.063407 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:16.063520 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:16.065744 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:16.065748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:16.065748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:16.065763 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:16.067091 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:16.067481 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:16.067483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:16.067484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:16.067564 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:16.067765 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:16.067766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:16.067767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:16.067976 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:16.068116 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:16.068216 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 22:08:16.068356 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260923 22:08:16.083317 7261 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } }
I20260923 22:08:16.083456 7261 leader_election.cc:302] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c2e7f371764beaae77f018ae14f5ec; no voters:
I20260923 22:08:16.083643 7261 leader_election.cc:288] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:16.083674 7266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:16.083818 7266 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [term 1 LEADER]: Becoming Leader. State: Replica: b2c2e7f371764beaae77f018ae14f5ec, State: Running, Role: LEADER
I20260923 22:08:16.083887 7261 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:16.083915 7266 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } }
I20260923 22:08:16.084203 7268 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c2e7f371764beaae77f018ae14f5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } } }
I20260923 22:08:16.084280 7268 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:16.084491 7267 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c2e7f371764beaae77f018ae14f5ec. Latest consensus state: current_term: 1 leader_uuid: "b2c2e7f371764beaae77f018ae14f5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c2e7f371764beaae77f018ae14f5ec" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46757 } } }
I20260923 22:08:16.084833 7267 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:16.085122 7271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:16.085559 7271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:16.087482 7271 catalog_manager.cc:1347] Generated new cluster ID: 1101f52921cf4feea60ccc8ecc33cbba
I20260923 22:08:16.087540 7271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:16.104243 7271 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:16.104799 7271 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:16.109052 7271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c2e7f371764beaae77f018ae14f5ec: Generated new TSK 0
I20260923 22:08:16.109208 7271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
W20260923 22:08:17.030094 1670 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.161.129: real 0.911s user 0.000s sys 0.003s
I20260923 22:08:17.030431 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:17.105753 7287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:17.105908 7287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:17.105927 7287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:17.107043 7287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:17.107081 7287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:17.107107 7287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:17.108596 7287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:17.108840 7287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:17.109036 7287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:17.113632 7292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:17.113627 7295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.113729 7287 server_base.cc:1034] running on GCE node
W20260923 22:08:17.113627 7293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:17.114181 7287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:17.114431 7287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:17.115592 7287 hybrid_clock.cc:648] HybridClock initialized: now 1790201297115580 us; error 25 us; skew 500 ppm
Sep 23 22:08:17 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:17.117861 7287 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:17.118202 7287 webserver.cc:460] Webserver started at http://127.1.161.129:39383/ using document root <none> and password file <none>
I20260923 22:08:17.118388 7287 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:17.118451 7287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:17.118579 7287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:17.119576 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "c6b077e15462496cb8acbeda454abe2d"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-npvd"
server_key: "47a8b7ed0a7859de34bf53598f620b0a"
server_key_iv: "ea9d84f9b07196687fce1f55c5d8c384"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.119922 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "c6b077e15462496cb8acbeda454abe2d"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-npvd"
server_key: "47a8b7ed0a7859de34bf53598f620b0a"
server_key_iv: "ea9d84f9b07196687fce1f55c5d8c384"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.121094 7287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:17.121742 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.121893 7287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:17.121989 7287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "c6b077e15462496cb8acbeda454abe2d"
format_stamp: "Formatted at 2026-09-23 22:08:17 on dist-test-slave-npvd"
server_key: "47a8b7ed0a7859de34bf53598f620b0a"
server_key_iv: "ea9d84f9b07196687fce1f55c5d8c384"
server_key_version: "encryptionkey@0"
I20260923 22:08:17.122059 7287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:17.131383 7287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:17.132973 7287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:17.133131 7287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:17.133340 7287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:17.133625 7287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:17.133701 7287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.133742 7287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:17.133772 7287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:17.139053 7287 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37567
I20260923 22:08:17.139096 7415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37567 every 8 connection(s)
I20260923 22:08:17.139362 7287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:17 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201297, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:17.144765 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7287
I20260923 22:08:17.144860 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:17.145073 1670 external_mini_cluster.cc:1428] Setting key 6d829dc7205273f41e957973a5482120
I20260923 22:08:17.148470 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.139989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:42365 (local address 127.1.161.190:46757)
0923 22:08:17.140141 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:17.140143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:17.140174 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:17.141045 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:17.141046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:17.141057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:17.141117 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:17.141673 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.142223 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.143064 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.143298 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.143345 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.145356 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:17.145357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:17.145357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:17.145363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:17.146936 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.147516 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.147517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.147517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.147569 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:17.147879 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:17.147879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:17.147880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:17.147986 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:17.148037 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:17.148360 (+ 323us) server_negotiation.cc:299] Negotiation successful
0923 22:08:17.148414 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260923 22:08:17.148715 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:17.139918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46757 (local address 127.1.161.129:42365)
0923 22:08:17.140063 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:17.140120 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:17.140930 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:17.141202 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:17.141210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:17.141435 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:17.141598 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.141602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.142303 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.142304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.142995 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:17.142997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:17.143352 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:17.143353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:17.143440 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:17.143769 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:17.143775 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:17.145248 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:17.147059 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.147065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.147068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.147439 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.147650 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:17.147651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:17.147652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:17.147785 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:17.148055 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:17.148060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:17.148212 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:08:17.148340 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 22:08:17.148493 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260923 22:08:17.149159 7416 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46757
I20260923 22:08:17.149246 7416 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:17.149405 7416 heartbeater.cc:511] Master 127.1.161.190:46757 requested a full tablet report, sending...
I20260923 22:08:17.149865 7225 ts_manager.cc:194] Registered new tserver with Master: c6b077e15462496cb8acbeda454abe2d (127.1.161.129:37567)
I20260923 22:08:17.150355 7225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:42365
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
I20260923 22:08:18.151225 7416 heartbeater.cc:503] Master 127.1.161.190:46757 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
W20260923 22:08:18.350880 1670 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.161.130: real 1.205s user 0.000s sys 0.003s
I20260923 22:08:18.351253 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:18.427310 7422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:18.427562 7422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:18.427603 7422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:18.429206 7422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:18.429270 7422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:18.429298 7422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:18.430805 7422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:18.430971 7422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:18.431139 7422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:18.436151 7428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:18.436144 7427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:18.436411 7422 server_base.cc:1034] running on GCE node
W20260923 22:08:18.436144 7430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:18.436703 7422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:18.436929 7422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:18.438082 7422 hybrid_clock.cc:648] HybridClock initialized: now 1790201298438079 us; error 35 us; skew 500 ppm
Sep 23 22:08:18 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:18.440510 7422 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:18.440936 7422 webserver.cc:460] Webserver started at http://127.1.161.130:40495/ using document root <none> and password file <none>
I20260923 22:08:18.441139 7422 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:18.441205 7422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:18.441325 7422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:18.442221 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "1f59495ad406457380e2a723d4924649"
format_stamp: "Formatted at 2026-09-23 22:08:18 on dist-test-slave-npvd"
server_key: "b82979794be080709335838e2645f7e0"
server_key_iv: "3bb5281ff1fa06887762f90ac86333bd"
server_key_version: "encryptionkey@0"
I20260923 22:08:18.442480 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "1f59495ad406457380e2a723d4924649"
format_stamp: "Formatted at 2026-09-23 22:08:18 on dist-test-slave-npvd"
server_key: "b82979794be080709335838e2645f7e0"
server_key_iv: "3bb5281ff1fa06887762f90ac86333bd"
server_key_version: "encryptionkey@0"
I20260923 22:08:18.443624 7422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:18.444329 7437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.444515 7422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:18.444583 7422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "1f59495ad406457380e2a723d4924649"
format_stamp: "Formatted at 2026-09-23 22:08:18 on dist-test-slave-npvd"
server_key: "b82979794be080709335838e2645f7e0"
server_key_iv: "3bb5281ff1fa06887762f90ac86333bd"
server_key_version: "encryptionkey@0"
I20260923 22:08:18.444631 7422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:18.473225 7422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:18.474920 7422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:18.475020 7422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:18.475251 7422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:18.475540 7422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:18.475577 7422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.475598 7422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:18.475611 7422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:18.480878 7422 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:44047
I20260923 22:08:18.480937 7550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:44047 every 8 connection(s)
I20260923 22:08:18.481244 7422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:18.485819 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7422
I20260923 22:08:18.485936 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:18.486155 1670 external_mini_cluster.cc:1428] Setting key 9203535361caaa5ab91fa9a40c6fddca
Sep 23 22:08:18 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:18.491346 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.481862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46757 (local address 127.1.161.130:53005)
0923 22:08:18.482108 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:18.482153 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:18.483008 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:18.483316 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:18.483319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:18.483558 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:18.483716 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.483720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.484534 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.484535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.485412 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:18.485414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.485855 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.485856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:18.486030 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.486511 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:18.486519 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:18.487908 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:18.490095 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:18.490101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:18.490103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:18.490306 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:18.490500 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:18.490500 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:18.490501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:18.490579 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:18.490832 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:18.490835 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:18.490926 (+ 91us) client_negotiation.cc:769] Sending connection context
0923 22:08:18.491014 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:08:18.491117 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260923 22:08:18.491348 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:18.481911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:53005 (local address 127.1.161.190:46757)
0923 22:08:18.482049 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:18.482051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:18.482249 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:08:18.483094 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:18.483095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:18.483105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:18.483254 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:18.483846 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.484439 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.485520 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:18.485752 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:18.485843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:18.488428 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:18.488430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:18.488430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:18.488436 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:18.489998 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:18.490380 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:18.490381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:18.490382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:18.490439 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:18.490640 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:18.490641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:18.490641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:18.490770 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:18.490838 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:18.491070 (+ 232us) server_negotiation.cc:299] Negotiation successful
0923 22:08:18.491161 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260923 22:08:18.491848 7551 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46757
I20260923 22:08:18.491952 7551 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:18.492141 7551 heartbeater.cc:511] Master 127.1.161.190:46757 requested a full tablet report, sending...
I20260923 22:08:18.492542 7225 ts_manager.cc:194] Registered new tserver with Master: 1f59495ad406457380e2a723d4924649 (127.1.161.130:44047)
I20260923 22:08:18.493036 7225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:53005
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
I20260923 22:08:19.493790 7551 heartbeater.cc:503] Master 127.1.161.190:46757 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
W20260923 22:08:19.740018 1670 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.161.131: real 1.253s user 0.000s sys 0.003s
I20260923 22:08:19.740373 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:19.815922 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:19.816083 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:19.816102 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:19.817081 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:19.817117 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:19.817139 7557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:19.818357 7557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:19.818555 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:19.818749 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:19.823109 7563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:19.823127 7565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:19.823220 7562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:19.823185 7557 server_base.cc:1034] running on GCE node
I20260923 22:08:19.823575 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:19.823772 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:19.824936 7557 hybrid_clock.cc:648] HybridClock initialized: now 1790201299824910 us; error 41 us; skew 500 ppm
Sep 23 22:08:19 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:19.827148 7557 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:19.827507 7557 webserver.cc:460] Webserver started at http://127.1.161.131:34661/ using document root <none> and password file <none>
I20260923 22:08:19.827651 7557 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:19.827692 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:19.827762 7557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:19.828526 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "7aaaa8c3fe7f4998bdb3df699e04db3a"
format_stamp: "Formatted at 2026-09-23 22:08:19 on dist-test-slave-npvd"
server_key: "3afc9de2dc36e7b9d54ad53b4aa692a3"
server_key_iv: "54c4f81bf9b9b826dceef88b3166376d"
server_key_version: "encryptionkey@0"
I20260923 22:08:19.828769 7557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "7aaaa8c3fe7f4998bdb3df699e04db3a"
format_stamp: "Formatted at 2026-09-23 22:08:19 on dist-test-slave-npvd"
server_key: "3afc9de2dc36e7b9d54ad53b4aa692a3"
server_key_iv: "54c4f81bf9b9b826dceef88b3166376d"
server_key_version: "encryptionkey@0"
I20260923 22:08:19.829795 7557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:19.830336 7572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:19.830487 7557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:19.830572 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "7aaaa8c3fe7f4998bdb3df699e04db3a"
format_stamp: "Formatted at 2026-09-23 22:08:19 on dist-test-slave-npvd"
server_key: "3afc9de2dc36e7b9d54ad53b4aa692a3"
server_key_iv: "54c4f81bf9b9b826dceef88b3166376d"
server_key_version: "encryptionkey@0"
I20260923 22:08:19.830642 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:19.842118 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:19.844450 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:19.844619 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:19.844826 7557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:19.845113 7557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:19.845170 7557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:19.845211 7557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:19.845237 7557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:19.850764 7557 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:40281
I20260923 22:08:19.850828 7685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:40281 every 8 connection(s)
I20260923 22:08:19.851083 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:19.854494 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7557
I20260923 22:08:19.854590 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:19.854755 1670 external_mini_cluster.cc:1428] Setting key 10d6b7c8f61ccd93ff60ff11608cb889
Sep 23 22:08:19 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:19.860076 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:19.851745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:38581 (local address 127.1.161.190:46757)
0923 22:08:19.851872 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:19.851874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:19.851986 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:08:19.852833 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:19.852834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:19.852849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:19.852912 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:19.853476 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.853982 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.855436 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.855719 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.855789 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.857693 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:19.857694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:19.857694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:19.857700 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:19.859012 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:19.859428 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:19.859429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:19.859429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:19.859482 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:19.859676 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:19.859677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:19.859677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:19.859753 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:19.859791 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:19.859968 (+ 177us) server_negotiation.cc:299] Negotiation successful
0923 22:08:19.860020 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260923 22:08:19.860245 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:19.851713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46757 (local address 127.1.161.131:38581)
0923 22:08:19.851873 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:19.851909 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:19.852719 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:19.852971 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:19.852975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:19.853209 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:19.853377 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.853381 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.854111 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.854112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.855348 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:19.855351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:19.855830 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:19.855831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:19.855968 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:19.856363 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:19.856370 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:19.857605 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:19.859087 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:19.859092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:19.859094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:19.859362 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:19.859547 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:19.859548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:19.859548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:19.859642 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:19.859800 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:19.859803 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:19.859894 (+ 91us) client_negotiation.cc:769] Sending connection context
0923 22:08:19.859959 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:08:19.860047 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260923 22:08:19.860689 7686 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46757
I20260923 22:08:19.860778 7686 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:19.860932 7686 heartbeater.cc:511] Master 127.1.161.190:46757 requested a full tablet report, sending...
I20260923 22:08:19.861245 7225 ts_manager.cc:194] Registered new tserver with Master: 7aaaa8c3fe7f4998bdb3df699e04db3a (127.1.161.131:40281)
I20260923 22:08:19.861644 7225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:38581
I20260923 22:08:19.866734 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/test-admin.keytab.
W20260923 22:08:20.480602 1670 mini_kdc.cc:302] Time spent creating keytab for test-admin: real 0.613s user 0.000s sys 0.001s
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.482328 1670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 22:08:20.483946 7694 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.483148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.1.161.129:37567)
0923 22:08:20.483332 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.483337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.483348 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:20.483377 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.483379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.483390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.483480 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.483832 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44524: BlockingRecv error: recv got EOF from 127.0.0.1:44524 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
W20260923 22:08:20.484023 1670 init.cc:244] Error while opening credential cache '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc)
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.484887 1670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.485776 1670 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:08:20.492501 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.486279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44530 (local address 127.1.161.129:37567)
0923 22:08:20.486339 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.486341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.486349 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:20.486404 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.486405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.486413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.486477 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.486832 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.487497 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.488132 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.488397 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.488479 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.490197 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:20.490204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:20.490205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:20.490212 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:20.491493 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.491811 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.491814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.491815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.491864 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.492041 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.492042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.492042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.492157 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:20.492286 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:20.492356 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:08:20.492422 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260923 22:08:20.494246 7694 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.493773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.1.161.129:37567)
0923 22:08:20.493806 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.493808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.493860 (+ 52us) server_negotiation.cc:407] Connection header received
0923 22:08:20.493906 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.493907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.493915 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.493955 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.494206 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44542: BlockingRecv error: recv got EOF from 127.0.0.1:44542 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:08:20.494324 1670 init.cc:244] Error while opening credential cache '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc)
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.495401 1670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.496258 1670 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:08:20.502058 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.496716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.1.161.129:37567)
0923 22:08:20.496771 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.496773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.496841 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:08:20.496877 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.496878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.496886 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.496947 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.497300 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.497705 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.498328 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.498506 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.498575 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.500247 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:20.500248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:20.500248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:20.500253 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:20.501206 (+ 953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.501487 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.501488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.501488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.501528 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.501675 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.501676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.501676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.501752 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:20.501812 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:20.501952 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:08:20.502002 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260923 22:08:20.503250 7694 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:20.502791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.1.161.129:37567)
0923 22:08:20.502842 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.502843 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.502850 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:20.502902 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.502903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.502917 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.502954 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.503212 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44546: BlockingRecv error: recv got EOF from 127.0.0.1:44546 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 22:08:20.503340 1670 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790201274647240-1670-0/krb5kdc/krb5cc' of type FILE
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.504133 1670 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:20.504974 1670 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.129@KRBTEST.COM
I20260923 22:08:20.510607 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:20.505466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.1.161.129:37567)
0923 22:08:20.505518 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:20.505520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:20.505610 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:08:20.505653 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:20.505653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:20.505662 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:20.505730 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:20.506077 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.506439 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.507012 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:20.507211 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:20.507263 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:20.508805 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:20.508806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:20.508806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:20.508812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:20.509731 (+ 919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.510021 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.510022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.510022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.510069 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:20.510230 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:20.510230 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:20.510231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:20.510306 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:20.510386 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:20.510477 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:08:20.510535 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:20.511229 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7287
I20260923 22:08:20.516274 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7422
I20260923 22:08:20.519982 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7557
I20260923 22:08:20.523567 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7193
2026-09-23T22:08:20Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (5586 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7707](info): set up 2 sockets
Sep 23 22:08:20 dist-test-slave-npvd krb5kdc[7707](info): commencing operation
krb5kdc: starting...
W20260923 22:08:21.442970 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.901s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab.
Entry for principal oryx/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab.
I20260923 22:08:21.547935 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38131
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab
--principal=oryx/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:21.617105 7720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:21.617275 7720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:21.617295 7720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:21.618324 7720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:21.618360 7720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:21.618373 7720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:21.618423 7720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:21.618450 7720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:21.619685 7720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab
--principal=oryx/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:21.619895 7720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:21.620117 7720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:21.624193 7725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:21.624176 7726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:21.624167 7728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:21.624428 7720 server_base.cc:1034] running on GCE node
I20260923 22:08:21.624665 7720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:21.624935 7720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:21.626089 7720 hybrid_clock.cc:648] HybridClock initialized: now 1790201301626079 us; error 27 us; skew 500 ppm
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:21.631587 7720 init.cc:377] Logged in from keytab as oryx/127.1.161.190@KRBTEST.COM (short username oryx)
I20260923 22:08:21.632025 7720 webserver.cc:460] Webserver started at http://127.1.161.190:46267/ using document root <none> and password file <none>
I20260923 22:08:21.632242 7720 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:21.632313 7720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:21.632465 7720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:21.633319 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "58f79a1786984d55a52d7e6a68f85969"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8a88b5ea852f3019a2e5a64e5963a63b"
server_key_iv: "8ba65a9e85fae5e786d9042ad85db950"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.633630 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "58f79a1786984d55a52d7e6a68f85969"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8a88b5ea852f3019a2e5a64e5963a63b"
server_key_iv: "8ba65a9e85fae5e786d9042ad85db950"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.634829 7720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:21.635628 7735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.635798 7720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.635902 7720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "58f79a1786984d55a52d7e6a68f85969"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8a88b5ea852f3019a2e5a64e5963a63b"
server_key_iv: "8ba65a9e85fae5e786d9042ad85db950"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.635990 7720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:21.646509 7720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:21.648130 7720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:21.648288 7720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:21.651821 7720 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38131
I20260923 22:08:21.651852 7787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38131 every 8 connection(s)
I20260923 22:08:21.652139 7720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:21.652951 7788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:21.654989 7788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Bootstrap starting.
I20260923 22:08:21.655604 7788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:21.655906 7788 log.cc:824] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:21.656548 7788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: No bootstrap required, opened a new log
I20260923 22:08:21.657892 7788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } }
I20260923 22:08:21.657999 7788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:21.658018 7788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f79a1786984d55a52d7e6a68f85969, State: Initialized, Role: FOLLOWER
I20260923 22:08:21.658172 7788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } }
I20260923 22:08:21.658227 7788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:21.658244 7788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:21.658304 7788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:21.658958 7788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } }
I20260923 22:08:21.659070 7788 leader_election.cc:302] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58f79a1786984d55a52d7e6a68f85969; no voters:
I20260923 22:08:21.659245 7788 leader_election.cc:288] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:21.659267 7791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:21.659410 7791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 LEADER]: Becoming Leader. State: Replica: 58f79a1786984d55a52d7e6a68f85969, State: Running, Role: LEADER
I20260923 22:08:21.659482 7788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:21.659548 7791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } }
I20260923 22:08:21.659914 7793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f79a1786984d55a52d7e6a68f85969. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } }
I20260923 22:08:21.660012 7793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:21.659900 7792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } }
I20260923 22:08:21.660274 7792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:21.660188 7800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:21.660724 7800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:21.661664 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7720
I20260923 22:08:21.661753 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:21.661924 1670 external_mini_cluster.cc:1428] Setting key a0a29fc0af051a3388cf8c6473498c11
I20260923 22:08:21.661978 7800 catalog_manager.cc:1347] Generated new cluster ID: 582e90cbf8a14b54ad7a7a6f449e80b7
I20260923 22:08:21.662014 7800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:21.671288 7800 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:21.671706 7800 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:21.673424 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.663269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.1.161.190:38131)
0923 22:08:21.663418 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.663425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.663452 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:21.663541 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.663547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.663590 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.663702 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.664372 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.665698 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.666771 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.667450 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.667534 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.669973 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:21.669977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:21.669977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:21.669986 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:21.671740 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.672096 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.672099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.672099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.672187 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.672361 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.672362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.672362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.672613 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:21.672718 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:21.673086 (+ 368us) server_negotiation.cc:299] Negotiation successful
0923 22:08:21.673210 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260923 22:08:21.674870 7800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Generated new TSK 0
I20260923 22:08:21.675025 7800 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
I20260923 22:08:21.697994 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:21.768657 7814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:21.768927 7814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:21.768968 7814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:21.770087 7814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:21.770165 7814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:21.770207 7814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:21.771618 7814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:21.771879 7814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:21.772097 7814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:21.777401 7814 server_base.cc:1034] running on GCE node
W20260923 22:08:21.777359 7819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:21.777367 7822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:21.777346 7820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:21.777750 7814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:21.777978 7814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:21.779143 7814 hybrid_clock.cc:648] HybridClock initialized: now 1790201301779156 us; error 45 us; skew 500 ppm
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:21.781340 7814 init.cc:377] Logged in from keytab as oryx/127.1.161.129@KRBTEST.COM (short username oryx)
I20260923 22:08:21.781760 7814 webserver.cc:460] Webserver started at http://127.1.161.129:46523/ using document root <none> and password file <none>
I20260923 22:08:21.781957 7814 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:21.782020 7814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:21.782136 7814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:21.782989 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "6c49f2022bdf46ea82a080009276e4c5"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "48118adbfa9b923a966503c5ced75543"
server_key_iv: "fadc2fb20f2b09d43df70da8086a519b"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.783325 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "6c49f2022bdf46ea82a080009276e4c5"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "48118adbfa9b923a966503c5ced75543"
server_key_iv: "fadc2fb20f2b09d43df70da8086a519b"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.784492 7814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:21.785203 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.785404 7814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:21.785506 7814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "6c49f2022bdf46ea82a080009276e4c5"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "48118adbfa9b923a966503c5ced75543"
server_key_iv: "fadc2fb20f2b09d43df70da8086a519b"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.785598 7814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:21.812282 7814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:21.813850 7814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:21.814002 7814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:21.814203 7814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:21.814517 7814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:21.814579 7814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.814620 7814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:21.814661 7814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.819865 7814 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:33689
I20260923 22:08:21.819943 7942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:33689 every 8 connection(s)
I20260923 22:08:21.820242 7814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:21.822154 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7814
I20260923 22:08:21.822248 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:21.822431 1670 external_mini_cluster.cc:1428] Setting key 623ba0f1d0b1b810bc4f29efe4fd7f69
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:21.829157 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.820946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:59033 (local address 127.1.161.190:38131)
0923 22:08:21.820989 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.820991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.821201 (+ 210us) server_negotiation.cc:407] Connection header received
0923 22:08:21.821588 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.821589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.821601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.821652 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.822639 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.823297 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.824166 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.824356 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.824432 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.826698 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:21.826699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:21.826700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:21.826707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:21.827940 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.828308 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.828309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.828309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.828362 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.828592 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.828593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.828593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.828694 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:21.828769 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:21.829002 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:08:21.829081 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:21.829303 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.820938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:59033)
0923 22:08:21.821075 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:21.821108 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:21.821507 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:21.821697 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:21.821699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:21.822321 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:21.822555 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.822561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.823372 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.823373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.824072 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.824074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.824442 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.824443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.824554 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.825046 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:21.825053 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:21.826612 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:21.828029 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:21.828036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:21.828038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:21.828233 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:21.828425 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:21.828426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:21.828426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:21.828526 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:21.828761 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:21.828766 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:21.828885 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 22:08:21.828981 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:08:21.829114 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260923 22:08:21.829726 7943 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:21.829793 7943 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:21.829953 7943 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.1.161.130@KRBTEST.COM; defaulting to no policy
I20260923 22:08:21.830454 7752 ts_manager.cc:194] Registered new tserver with Master: 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:21.830956 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:59033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
I20260923 22:08:21.846702 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:21.918798 7948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:21.918982 7948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:21.919013 7948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:21.920018 7948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:21.920079 7948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:21.920116 7948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:21.921304 7948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:21.921528 7948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:21.921725 7948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:21.926147 7956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:21.926288 7948 server_base.cc:1034] running on GCE node
W20260923 22:08:21.926147 7953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:21.926224 7954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:21.926523 7948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:21.926748 7948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:21.927875 7948 hybrid_clock.cc:648] HybridClock initialized: now 1790201301927863 us; error 26 us; skew 500 ppm
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:21.929978 7948 init.cc:377] Logged in from keytab as oryx/127.1.161.130@KRBTEST.COM (short username oryx)
I20260923 22:08:21.930392 7948 webserver.cc:460] Webserver started at http://127.1.161.130:34581/ using document root <none> and password file <none>
I20260923 22:08:21.930598 7948 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:21.930665 7948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:21.930795 7948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:21.931648 7948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "56fe4644f0f84222a0ac1d6b641cb2ab"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8e3ae02b0216844434637e6cde8b6f9c"
server_key_iv: "aa840df3d1aaed4361ac0f2484216f36"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.931991 7948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "56fe4644f0f84222a0ac1d6b641cb2ab"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8e3ae02b0216844434637e6cde8b6f9c"
server_key_iv: "aa840df3d1aaed4361ac0f2484216f36"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.933143 7948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:21.933751 7963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.933900 7948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:21.933983 7948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "56fe4644f0f84222a0ac1d6b641cb2ab"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8e3ae02b0216844434637e6cde8b6f9c"
server_key_iv: "aa840df3d1aaed4361ac0f2484216f36"
server_key_version: "encryptionkey@0"
I20260923 22:08:21.934051 7948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:21.954046 7948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:21.955701 7948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:21.955856 7948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:21.956058 7948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:21.956326 7948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:21.956375 7948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.956408 7948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:21.956432 7948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:21.961498 7948 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:44831
I20260923 22:08:21.961540 8076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:44831 every 8 connection(s)
I20260923 22:08:21.961773 7948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:21 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:21.969897 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.962350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:39739 (local address 127.1.161.190:38131)
0923 22:08:21.962405 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:21.962406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:21.962559 (+ 153us) server_negotiation.cc:407] Connection header received
0923 22:08:21.962962 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:21.962963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:21.962974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:21.963013 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:21.963827 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.964284 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.965188 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.965382 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.965471 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.967665 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:21.967666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:21.967666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:21.967672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:21.968660 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.969152 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.969153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.969153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.969195 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:21.969410 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:21.969411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:21.969411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:21.969509 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:21.969562 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:21.969785 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 22:08:21.969843 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:21.970021 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:21.962326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:39739)
0923 22:08:21.962454 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:21.962491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:21.962864 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:21.963078 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:21.963081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:21.963586 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:21.963730 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.963734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.964361 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.964362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.965070 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:21.965072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:21.965505 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:21.965506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:21.965620 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:21.966132 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:21.966139 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:21.967518 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:21.968877 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:21.968882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:21.968883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:21.969085 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:21.969253 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:21.969254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:21.969254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:21.969339 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:21.969572 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:21.969575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:21.969684 (+ 109us) client_negotiation.cc:769] Sending connection context
0923 22:08:21.969759 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:08:21.969855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260923 22:08:21.970456 8077 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:21.970546 8077 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:21.970670 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 7948
I20260923 22:08:21.970690 8077 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
I20260923 22:08:21.970767 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:21.970957 1670 external_mini_cluster.cc:1428] Setting key a410ca01283cae6e1e495446f4a145b6
I20260923 22:08:21.971062 7752 ts_manager.cc:194] Registered new tserver with Master: 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831)
I20260923 22:08:21.971568 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:39739
WARNING: no policy specified for oryx/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
I20260923 22:08:21.994540 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:22.064049 8082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.064234 8082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.064266 8082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.065299 8082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.065371 8082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.065414 8082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:22.066711 8082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:38131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:22.066927 8082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.067114 8082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.071368 8090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.071382 8087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.071379 8088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.071442 8082 server_base.cc:1034] running on GCE node
I20260923 22:08:22.071663 8082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.071877 8082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.073011 8082 hybrid_clock.cc:648] HybridClock initialized: now 1790201302073012 us; error 38 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.075102 8082 init.cc:377] Logged in from keytab as oryx/127.1.161.131@KRBTEST.COM (short username oryx)
I20260923 22:08:22.075511 8082 webserver.cc:460] Webserver started at http://127.1.161.131:33333/ using document root <none> and password file <none>
I20260923 22:08:22.075729 8082 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.075799 8082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.075920 8082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:22.076774 8082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "b7f6337539c245fbae3be625407f769a"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "19f36a4be865eeb0e193a5d76b7691c7"
server_key_iv: "65b54f78de4029252e88cedce9793d04"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.077078 8082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "b7f6337539c245fbae3be625407f769a"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "19f36a4be865eeb0e193a5d76b7691c7"
server_key_iv: "65b54f78de4029252e88cedce9793d04"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.078178 8082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:22.078799 8097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.078953 8082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:22.079036 8082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b7f6337539c245fbae3be625407f769a"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "19f36a4be865eeb0e193a5d76b7691c7"
server_key_iv: "65b54f78de4029252e88cedce9793d04"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.079109 8082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.084525 8082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.086067 8082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.086230 8082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.086426 8082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:22.086658 8082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:22.086714 8082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.086747 8082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:22.086771 8082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.091591 8082 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:46331
I20260923 22:08:22.091619 8210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:46331 every 8 connection(s)
I20260923 22:08:22.091864 8082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:22.098291 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8082
I20260923 22:08:22.098400 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:22.098556 1670 external_mini_cluster.cc:1428] Setting key 33d94061c24fc49acbb98ffd415cbbed
I20260923 22:08:22.099850 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.092466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:35055 (local address 127.1.161.190:38131)
0923 22:08:22.092507 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.092525 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.092692 (+ 167us) server_negotiation.cc:407] Connection header received
0923 22:08:22.093178 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.093179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.093191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.093234 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.093966 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.094426 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.095460 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.095654 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.095767 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.097519 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.097520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.097521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.097527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.098680 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.099095 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.099096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.099096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.099158 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.099418 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.099419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.099419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.099508 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.099553 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.099744 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.099798 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:22.100013 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.092430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:35055)
0923 22:08:22.092577 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.092612 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.093079 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.093285 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.093288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.093736 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.093878 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.093881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.094522 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.094523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.095373 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.095375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.095772 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.095773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.095896 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.096299 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.096306 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.097431 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.098780 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.098786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.098788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.099033 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.099236 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.099237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.099237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.099354 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.099568 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.099572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.099656 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.099733 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.099824 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260923 22:08:22.100415 8211 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:22.100494 8211 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.100651 8211 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
I20260923 22:08:22.100984 7752 ts_manager.cc:194] Registered new tserver with Master: b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:22.101269 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:35055
I20260923 22:08:22.110086 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.161.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.189.keytab.
Entry for principal oryx/127.1.161.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.189.keytab.
I20260923 22:08:22.133546 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:45695
--webserver_interface=127.1.161.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.189.keytab
--principal=oryx/127.1.161.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38131,127.1.161.189:45695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:22.204733 8216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.204926 8216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.204954 8216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.206079 8216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:22.206153 8216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.206179 8216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.206200 8216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:22.206226 8216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:22.207458 8216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38131,127.1.161.189:45695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.189.keytab
--principal=oryx/127.1.161.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.161.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
W20260923 22:08:22.207571 8216 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.161.190:38131,127.1.161.189:45695'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 22:08:22.207710 8216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.207894 8216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.211987 8224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.211997 8221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.212010 8216 server_base.cc:1034] running on GCE node
W20260923 22:08:22.211998 8222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.212390 8216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.212728 8216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.213882 8216 hybrid_clock.cc:648] HybridClock initialized: now 1790201302213880 us; error 34 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.216265 8216 init.cc:377] Logged in from keytab as oryx/127.1.161.189@KRBTEST.COM (short username oryx)
I20260923 22:08:22.216598 8216 webserver.cc:460] Webserver started at http://127.1.161.189:44983/ using document root <none> and password file <none>
I20260923 22:08:22.216796 8216 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.216858 8216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.216988 8216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:22.217898 8216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/instance:
uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "a5f62738b17379039ffdec6b0981779e"
server_key_iv: "3c917200e02c48a996628839a601dcaf"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.218216 8216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal/instance:
uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "a5f62738b17379039ffdec6b0981779e"
server_key_iv: "3c917200e02c48a996628839a601dcaf"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.219333 8216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:22.219913 8231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.220060 8216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.220142 8216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "a5f62738b17379039ffdec6b0981779e"
server_key_iv: "3c917200e02c48a996628839a601dcaf"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.220234 8216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.233886 8216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.235584 8216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.235731 8216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.238914 8216 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.189:45695
I20260923 22:08:22.238955 8283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.189:45695 every 8 connection(s)
I20260923 22:08:22.239269 8216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
I20260923 22:08:22.240032 8284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:22.241236 8284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.189@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:22.247314 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8216
I20260923 22:08:22.248762 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.241796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.1.161.190:38131)
0923 22:08:22.241857 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.241859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.242035 (+ 176us) server_negotiation.cc:407] Connection header received
0923 22:08:22.242385 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.242386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.242396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.242443 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.242897 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.243405 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.244208 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.244362 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.244416 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.246286 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.246287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.246288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.246294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.247350 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.247795 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.247796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.247796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.247851 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.248113 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.248114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.248114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.248237 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.248284 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.248589 (+ 305us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.248677 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:22.248782 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.241784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45570)
0923 22:08:22.241944 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.241970 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.242306 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.242493 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.242495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.242676 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.242828 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.242831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.243470 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.243471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.244125 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.244126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.244432 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.244432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.244510 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.244900 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.244908 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.246182 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.247437 (+ 1255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.247442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.247444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.247709 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.247915 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.247916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.247916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.248045 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.248300 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.248306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.248442 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.248544 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.248663 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260923 22:08:22.249289 8284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.189@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
I20260923 22:08:22.255497 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.247714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.1.161.189:45695)
0923 22:08:22.247849 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.247853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.247870 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:08:22.247919 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.247921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.247931 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.247989 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.248582 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.249418 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.250248 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.250592 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.250634 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.252752 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.252758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.252758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.252764 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.254206 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.254604 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.254606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.254606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.254658 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.254862 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.254862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.254863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.254970 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.255069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.255235 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.255359 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260923 22:08:22.256218 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.249454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46392)
0923 22:08:22.249484 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.249494 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.249531 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.249709 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.249710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.249825 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.249864 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.249865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.250453 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.250454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.251024 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.251026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.251372 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.251374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.251444 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.251728 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.251732 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.253025 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.255085 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.255086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.255086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.255359 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.255581 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.255582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.255582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.255641 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.255918 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.255919 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.255957 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.256035 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.256124 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:08:22.256348 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.249501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.1.161.189:45695)
0923 22:08:22.249603 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.249604 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.249608 (+ 4us) server_negotiation.cc:407] Connection header received
0923 22:08:22.249620 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.249620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.249626 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.249681 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.249929 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.250381 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.251093 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.251309 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.251353 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.253097 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.253098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.253098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.253104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.254986 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.255441 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.255442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.255442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.255509 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.255724 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.255725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.255725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.255846 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.255911 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.256054 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.256205 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":86,"thread_start_us":71,"threads_started":1}
I20260923 22:08:22.257265 8284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Bootstrap starting.
I20260923 22:08:22.257663 8284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:22.257865 8284 log.cc:824] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:22.258327 8284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: No bootstrap required, opened a new log
I20260923 22:08:22.259506 8284 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } }
I20260923 22:08:22.259667 8284 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:22.259714 8284 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd770ae25cd24f2882d85d8e09b4f69e, State: Initialized, Role: FOLLOWER
I20260923 22:08:22.259812 8284 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } }
I20260923 22:08:22.259992 8291 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } } }
I20260923 22:08:22.260048 8284 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:22.260061 8291 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:08:22.261379 8302 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:22.261425 8302 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:22.262732 8216 master_runner.cc:428] Detected that this master dd770ae25cd24f2882d85d8e09b4f69e is joining an existing cluster
I20260923 22:08:22.262785 8216 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.161.189:45695
I20260923 22:08:22.269728 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.263754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.1.161.189:45695)
0923 22:08:22.263812 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.263814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.263824 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:22.263841 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.263842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.263849 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.263899 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.264216 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.264687 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.265482 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.265813 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.265878 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.266852 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.266853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.266854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.266859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.268605 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269025 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.269026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.269027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.269077 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269249 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.269250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.269250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.269348 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.269431 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.269546 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.269655 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":42}
I20260923 22:08:22.269780 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.263877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.1.161.190:38131)
0923 22:08:22.263906 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.263907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.264098 (+ 191us) server_negotiation.cc:407] Connection header received
0923 22:08:22.264118 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.264118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.264127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.264168 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.264442 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.265047 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.265875 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.266075 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.266172 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.266987 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.266989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.266989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.266994 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.268705 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269079 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.269080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.269080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.269128 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269301 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.269302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.269302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.269419 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.269477 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.269633 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.269695 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 22:08:22.269893 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.263589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46404)
0923 22:08:22.263740 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.263749 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.263792 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.263929 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.263930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.264084 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.264133 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.264134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.264761 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.264762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.265392 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.265394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.265918 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.265918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.265972 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.266415 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.266419 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.266760 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.268696 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.268697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.268697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.268953 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.269128 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.269128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.269191 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.269424 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.269425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.269458 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.269514 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.269793 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":105,"threads_started":1}
I20260923 22:08:22.270278 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.263881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45582)
0923 22:08:22.264029 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.264036 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.264081 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.264170 (+ 89us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.264171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.264319 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.264362 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.264363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.265165 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.265166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.265786 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.265788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.266186 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.266187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.266265 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.266561 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.266565 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.266909 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.268773 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.268773 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.268774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.269020 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.269174 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.269174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.269175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.269220 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.269501 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.269502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.269536 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.269599 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.269725 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":28,"thread_start_us":86,"threads_started":1}
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.190@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
I20260923 22:08:22.278263 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.271971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46410)
0923 22:08:22.272092 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.272123 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.272180 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.272301 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.272303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.272581 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.272627 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.272629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.273117 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.273118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.273928 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.273931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.274216 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.274217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.274282 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.274709 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.274713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.276104 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.277459 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.277465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.277466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.277623 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.277799 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.277799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.277799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.277842 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.278043 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.278048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.278080 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.278139 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.278211 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":41,"threads_started":1}
I20260923 22:08:22.278309 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.272027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.1.161.189:45695)
0923 22:08:22.272054 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.272056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.272174 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:08:22.272250 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.272251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.272256 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.272295 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.272694 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.273045 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.274001 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.274146 (+ 145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.274186 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.276208 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.276209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.276210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.276215 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.277388 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.277690 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.277691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.277692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.277738 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.277892 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.277893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.277893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.277989 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.278043 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.278176 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.278241 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260923 22:08:22.278841 7752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.161.189:45695: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
member_type: NON_VOTER
last_known_addr {
host: "127.1.161.189"
port: 45695
}
attrs {
promote: true
}
}
dest_uuid: "58f79a1786984d55a52d7e6a68f85969"
cas_config_opid_index: -1
I20260923 22:08:22.279114 7752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } }
I20260923 22:08:22.279568 8312 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dd770ae25cd24f2882d85d8e09b4f69e (127.1.161.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } }
I20260923 22:08:22.279770 8258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:22.280351 8312 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.161.189:45695
I20260923 22:08:22.280493 8258 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 22:08:22.280560 8311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } }
I20260923 22:08:22.280622 8311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:22.280653 8216 master.cc:537] Master@127.1.161.189:45695 shutting down...
I20260923 22:08:22.280793 8312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:08:22.281433 7737 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 22:08:22.281836 7737 consensus_peers.cc:487] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 -> Peer dd770ae25cd24f2882d85d8e09b4f69e (127.1.161.189:45695): Couldn't send request to peer dd770ae25cd24f2882d85d8e09b4f69e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:22.282829 8216 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:22.282899 8216 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:22.282938 8216 tablet_replica.cc:330] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: stopping tablet replica
I20260923 22:08:22.294982 8216 master.cc:552] Master@127.1.161.189:45695 shutdown complete.
I20260923 22:08:22.295051 8216 master_runner.cc:305] Clearing existing system tablet
I20260923 22:08:22.295233 8216 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.295292 8216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.295861 8216 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:22.296329 8316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.296481 8216 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.296540 8216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.296562 8216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "a5f62738b17379039ffdec6b0981779e"
server_key_iv: "3c917200e02c48a996628839a601dcaf"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.297235 8216 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:22.298672 8216 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:08:22.298732 8216 log.cc:1196] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 22:08:22.298892 8216 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Deleting consensus metadata
I20260923 22:08:22.298991 8216 master_runner.cc:315] Copying system tablet from 127.1.161.190:38131
I20260923 22:08:22.299495 8216 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: tablet copy: Beginning tablet copy session from remote peer at address 127.1.161.190:38131
I20260923 22:08:22.304695 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.299726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45590)
0923 22:08:22.299818 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.299826 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.299877 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.299994 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.299994 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.300109 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.300150 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.300152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.300926 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.300927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.301371 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.301373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.301762 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.301763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.301801 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.302178 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.302182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.302401 (+ 219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.303921 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.303921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.303922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.304055 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.304229 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.304230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.304230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.304272 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.304489 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.304490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.304517 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.304559 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.304617 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":61,"threads_started":1}
I20260923 22:08:22.304710 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.299781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.1.161.190:38131)
0923 22:08:22.299820 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.299821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.299881 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:08:22.299932 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.299932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.299941 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.299991 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.300217 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.300858 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.301441 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.301698 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.301756 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.302477 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.302478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.302478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.302484 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.303853 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.304119 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.304120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.304120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.304171 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.304334 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.304334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.304335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.304438 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.304495 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.304577 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.304638 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:22.304944 7772 tablet_copy_service.cc:140] P 58f79a1786984d55a52d7e6a68f85969: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dd770ae25cd24f2882d85d8e09b4f69e ({username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:45590)
I20260923 22:08:22.305027 7772 tablet_copy_service.cc:161] P 58f79a1786984d55a52d7e6a68f85969: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dd770ae25cd24f2882d85d8e09b4f69e at {username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:45590: session id = dd770ae25cd24f2882d85d8e09b4f69e-00000000000000000000000000000000
I20260923 22:08:22.305588 7772 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:08:22.306149 8216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:08:22.307166 8216 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: tablet copy: Starting download of 0 data blocks...
I20260923 22:08:22.307354 8216 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: tablet copy: Starting download of 1 WAL segments...
I20260923 22:08:22.308051 8216 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:08:22.308804 7772 tablet_copy_service.cc:342] P 58f79a1786984d55a52d7e6a68f85969: Request end of tablet copy session dd770ae25cd24f2882d85d8e09b4f69e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:45590
I20260923 22:08:22.308867 7772 tablet_copy_service.cc:433] P 58f79a1786984d55a52d7e6a68f85969: ending tablet copy session dd770ae25cd24f2882d85d8e09b4f69e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dd770ae25cd24f2882d85d8e09b4f69e
I20260923 22:08:22.310983 8216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.312037 8323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.312153 8216 server_base.cc:1034] running on GCE node
W20260923 22:08:22.312120 8326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.315291 8324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.315474 8216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.315593 8216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.316725 8216 hybrid_clock.cc:648] HybridClock initialized: now 1790201302316720 us; error 31 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.317750 8216 init.cc:377] Logged in from keytab as oryx/127.1.161.189@KRBTEST.COM (short username oryx)
I20260923 22:08:22.318005 8216 webserver.cc:460] Webserver started at http://127.1.161.189:37621/ using document root <none> and password file <none>
I20260923 22:08:22.318115 8216 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.318173 8216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.318686 8216 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:22.319126 8333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.319260 8216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.319303 8216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "dd770ae25cd24f2882d85d8e09b4f69e"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "a5f62738b17379039ffdec6b0981779e"
server_key_iv: "3c917200e02c48a996628839a601dcaf"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.319350 8216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.324527 8216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.325912 8216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.328707 8216 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.189:45695
I20260923 22:08:22.329017 8385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.189:45695 every 8 connection(s)
I20260923 22:08:22.329051 8216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
I20260923 22:08:22.329716 8386 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:22.330063 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Bootstrap starting.
I20260923 22:08:22.332031 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:22.332245 8386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Bootstrap complete.
I20260923 22:08:22.332417 8386 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } }
I20260923 22:08:22.332482 8386 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd770ae25cd24f2882d85d8e09b4f69e, State: Initialized, Role: LEARNER
I20260923 22:08:22.332504 8386 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } }
I20260923 22:08:22.332677 8386 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:22.332685 8389 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } }
I20260923 22:08:22.332805 8389 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:22.333843 8400 catalog_manager.cc:1259] Loaded cluster ID: 582e90cbf8a14b54ad7a7a6f449e80b7
I20260923 22:08:22.333905 8400 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: loading cluster ID for follower catalog manager: success
I20260923 22:08:22.334821 8400 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: acquiring CA information for follower catalog manager: success
I20260923 22:08:22.335085 8400 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:08:22.363137 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.357012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.1.161.189:45695)
0923 22:08:22.357118 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.357120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.357127 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:22.357148 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.357148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.357155 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.357219 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.357717 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.358706 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.359390 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.359616 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.359722 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.360846 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.360848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.360848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.360854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.362114 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.362432 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.362433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.362434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.362495 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.362680 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.362681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.362681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.362775 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.362870 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.362958 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.363026 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":49,"threads_started":1}
WARNING: no policy specified for oryx/127.1.161.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.188.keytab.
Entry for principal oryx/127.1.161.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.188.keytab.
I20260923 22:08:22.386113 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:39677
--webserver_interface=127.1.161.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.188.keytab
--principal=oryx/127.1.161.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:22.457176 8404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.457338 8404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.457356 8404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.458308 8404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:22.458343 8404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.458353 8404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.458393 8404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:22.458416 8404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:22.459717 8404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.188.keytab
--principal=oryx/127.1.161.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:39677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.161.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:22.459931 8404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.460136 8404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.464952 8409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.464944 8412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.464944 8410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.464973 8404 server_base.cc:1034] running on GCE node
I20260923 22:08:22.465487 8404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.465795 8404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.466964 8404 hybrid_clock.cc:648] HybridClock initialized: now 1790201302466936 us; error 40 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.469440 8404 init.cc:377] Logged in from keytab as oryx/127.1.161.188@KRBTEST.COM (short username oryx)
I20260923 22:08:22.469835 8404 webserver.cc:460] Webserver started at http://127.1.161.188:39767/ using document root <none> and password file <none>
I20260923 22:08:22.470033 8404 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.470099 8404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.470213 8404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:22.471092 8404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/instance:
uuid: "95de76c2e6fc401da5ffa317182adc13"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "5c736edd4c0961366ef8f02f0107e94d"
server_key_iv: "a9826c033515b73b52c5e091e9bb4720"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.471485 8404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal/instance:
uuid: "95de76c2e6fc401da5ffa317182adc13"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "5c736edd4c0961366ef8f02f0107e94d"
server_key_iv: "a9826c033515b73b52c5e091e9bb4720"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.472604 8404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:22.473237 8419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.473424 8404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:22.473512 8404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "95de76c2e6fc401da5ffa317182adc13"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "5c736edd4c0961366ef8f02f0107e94d"
server_key_iv: "a9826c033515b73b52c5e091e9bb4720"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.473572 8404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.487443 8404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.489048 8404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.489229 8404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.493253 8404 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.188:39677
I20260923 22:08:22.493314 8471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.188:39677 every 8 connection(s)
I20260923 22:08:22.493607 8404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
I20260923 22:08:22.494788 8472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:22.496104 8472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:08:22.499971 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8404
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.188@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:22.503275 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.496783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.1.161.190:38131)
0923 22:08:22.496843 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.496845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.496875 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:08:22.497223 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.497224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.497232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.497276 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.497703 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.498105 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.498869 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.499024 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.499070 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.500906 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.500907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.500908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.500914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.502079 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.502434 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.502435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.502435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.502487 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.502717 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.502718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.502718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.502813 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.502876 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.503109 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.503199 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:08:22.503391 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.496645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45600)
0923 22:08:22.496782 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.496814 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.497153 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.497316 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.497318 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.497501 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.497643 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.497645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.498171 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.498171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.498801 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.498802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.499097 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.499098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.499233 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.499558 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.499564 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.500824 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.502170 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.502181 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.502183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.502364 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.502556 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.502556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.502556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.502648 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.502877 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.502883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.502991 (+ 108us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.503078 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.503214 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260923 22:08:22.503793 8472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.188@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
I20260923 22:08:22.507798 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.500412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59446 (local address 127.1.161.188:39677)
0923 22:08:22.500509 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.500513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.500619 (+ 106us) server_negotiation.cc:407] Connection header received
0923 22:08:22.500638 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.500640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.500664 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.500756 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.501244 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.501778 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.502401 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.502641 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.502718 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.505077 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.505083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.505084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.505089 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.506749 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.507076 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.507079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.507079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.507130 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.507327 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.507328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.507328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.507438 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.507560 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.507642 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.507701 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":55,"threads_started":1}
I20260923 22:08:22.509478 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.503984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46444)
0923 22:08:22.504025 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.504033 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.504062 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.504352 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.504353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.504444 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.504479 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.504480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.505172 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.505172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.505649 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.505650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.506040 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.506041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.506107 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.506473 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.506477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.507426 (+ 949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.508810 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.508810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.508811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.508937 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.509091 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.509091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.509092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.509132 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.509315 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.509315 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.509341 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.509383 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.509435 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:08:22.509496 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.504156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.1.161.189:45695)
0923 22:08:22.504221 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.504223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.504248 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:08:22.504271 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.504272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.504281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.504334 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.504561 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.505027 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.505745 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.505955 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.506040 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.507514 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.507515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.507515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.507522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.508743 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.508999 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.508999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.509000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.509040 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.509182 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.509182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.509183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.509268 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.509309 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.509397 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.509448 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:22.509874 8472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.188@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
I20260923 22:08:22.514310 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.510048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59454)
0923 22:08:22.510089 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.510098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.510128 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.510214 (+ 86us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.510214 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.510297 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.510317 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.510318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.510744 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.510745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.511103 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.511104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.511450 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.511450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.511481 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.511836 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.511840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.512688 (+ 848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.513696 (+ 1008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.513696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.513697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.513821 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.513966 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.513967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.513967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.514006 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.514142 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.514143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.514167 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.514214 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.514268 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 22:08:22.514441 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.510090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.1.161.188:39677)
0923 22:08:22.510115 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.510117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.510122 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:08:22.510170 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.510171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.510177 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.510212 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.510350 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.510694 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.511213 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.511394 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.511439 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.512749 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.512750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.512750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.512754 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.513636 (+ 882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.513879 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.513879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.513879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.513917 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.514038 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.514038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.514038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.514101 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.514136 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.514226 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.514303 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":13}
I20260923 22:08:22.515535 8472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Bootstrap starting.
I20260923 22:08:22.516232 8472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:22.516501 8472 log.cc:824] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:22.517221 8472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: No bootstrap required, opened a new log
I20260923 22:08:22.518517 8472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } }
I20260923 22:08:22.518628 8472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:22.518666 8472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95de76c2e6fc401da5ffa317182adc13, State: Initialized, Role: FOLLOWER
I20260923 22:08:22.518778 8472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } }
I20260923 22:08:22.518981 8477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } } }
I20260923 22:08:22.519032 8472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:22.519052 8477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:08:22.520342 8488 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:08:22.520390 8488 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:08:22.522225 8404 master_runner.cc:428] Detected that this master 95de76c2e6fc401da5ffa317182adc13 is joining an existing cluster
I20260923 22:08:22.522279 8404 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.161.188:39677
I20260923 22:08:22.528856 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.523527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.1.161.190:38131)
0923 22:08:22.523560 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.523562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.523687 (+ 125us) server_negotiation.cc:407] Connection header received
0923 22:08:22.523734 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.523735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.523743 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.523783 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.524147 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.524575 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.525235 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.525431 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.525514 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.526322 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.526324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.526324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.526330 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.527782 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.528103 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.528103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.528104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.528150 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.528353 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.528353 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.528353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.528466 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.528497 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.528712 (+ 215us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.528776 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:22.529453 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.523506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45616)
0923 22:08:22.523625 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.523633 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.523684 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.523784 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.523784 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.523917 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.524045 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.524047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.524637 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.524638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.525104 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.525106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.525527 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.525528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.525589 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.525969 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.525973 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.526218 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.527858 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.527858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.527859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.528034 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.528241 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.528241 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.528241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.528301 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.528531 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.528532 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.528578 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.529252 (+ 674us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.529345 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260923 22:08:22.532768 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.524092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59466)
0923 22:08:22.527265 (+ 3173us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.527275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.527356 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.527492 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.527493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.527807 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.527861 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.527862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.528609 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.528610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.529451 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.529454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.529892 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.529893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.529936 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.530304 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.530308 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.530611 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.531972 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.531972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.531973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.532103 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.532273 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.532273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.532273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.532311 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.532503 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.532503 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.532538 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.532591 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.532670 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3156,"thread_start_us":55,"threads_started":1}
I20260923 22:08:22.532825 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.524144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.1.161.188:39677)
0923 22:08:22.524172 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.524174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.527320 (+ 3146us) server_negotiation.cc:407] Connection header received
0923 22:08:22.527418 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.527418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.527425 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.527448 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.527924 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.528544 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.529537 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.529808 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.529840 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.530688 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.530688 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.530689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.530694 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.531904 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.532170 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.532171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.532171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.532219 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.532343 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.532344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.532344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.532437 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.532484 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.532624 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.532729 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":15}
I20260923 22:08:22.550503 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.523550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46454)
0923 22:08:22.527234 (+ 3684us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.527243 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.527308 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.527487 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.527487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.527767 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.527810 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.527811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.528619 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.528619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.529721 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.529723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.530188 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.530189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.530249 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.547733 (+ 17484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.547737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.548057 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.549532 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.549532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.549533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.549726 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.549917 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.549918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.549918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.549976 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.550196 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.550197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.550233 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.550327 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.550409 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3664,"thread_start_us":439,"threads_started":1}
I20260923 22:08:22.550508 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.524212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.1.161.189:45695)
0923 22:08:22.524261 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.524263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.527288 (+ 3025us) server_negotiation.cc:407] Connection header received
0923 22:08:22.527384 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.527385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.527393 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.527418 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.527886 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.528535 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.529822 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.530084 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.530130 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.548132 (+ 18002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.548134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.548134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.548140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.549455 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.549798 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.549799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.549799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.549848 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.550029 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.550030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.550030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.550134 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.550194 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.550354 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.550423 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201301, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.190@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
I20260923 22:08:22.554044 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.548211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59478)
0923 22:08:22.548264 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.548273 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.548306 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.548428 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.548429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.548579 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.548620 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.548622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.549343 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.549344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.549927 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.549929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.550278 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.550279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.550338 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.550685 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.550689 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.551902 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.553276 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.553276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.553277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.553405 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.553587 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.553587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.553587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.553627 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.553864 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.553865 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.553892 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.553934 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.553993 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:08:22.554087 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.548252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.1.161.188:39677)
0923 22:08:22.548294 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.548295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.548303 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:22.548362 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.548362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.548370 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.548426 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.548697 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.549258 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.550000 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.550230 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.550278 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.551983 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.551984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.551984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.551990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.553201 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.553472 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.553473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.553473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.553526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.553702 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.553702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.553703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.553805 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.553868 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.553956 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.554017 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:22.554500 7752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.161.188:39677: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "95de76c2e6fc401da5ffa317182adc13"
member_type: NON_VOTER
last_known_addr {
host: "127.1.161.188"
port: 39677
}
attrs {
promote: true
}
}
dest_uuid: "58f79a1786984d55a52d7e6a68f85969"
cas_config_opid_index: 5
I20260923 22:08:22.554745 7752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } }
I20260923 22:08:22.555202 8311 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.555334 8446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:22.556036 8311 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.161.188:39677
I20260923 22:08:22.556113 8311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.556182 8311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:22.556216 8446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 FOLLOWER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:08:22.556258 8404 master.cc:537] Master@127.1.161.188:39677 shutting down...
I20260923 22:08:22.556464 8312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:08:22.557097 7736 consensus_peers.cc:487] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 -> Peer 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188:39677): Couldn't send request to peer 95de76c2e6fc401da5ffa317182adc13. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:22.558815 8404 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:22.558883 8404 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:22.558916 8404 tablet_replica.cc:330] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: stopping tablet replica
W20260923 22:08:22.559967 8421 connection.cc:441] server connection from 127.0.0.1:59446 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:59446 (request call id 11) could send its response
I20260923 22:08:22.560286 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.555234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.1.161.189:45695)
0923 22:08:22.556284 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.556286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.556646 (+ 360us) server_negotiation.cc:407] Connection header received
0923 22:08:22.556665 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.556666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.556695 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.556749 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:22.556903 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.557494 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.559259 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.559743 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.559798 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.559863 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.560145 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.560181 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033}
I20260923 22:08:22.560393 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.555141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.0.0.1:46456)
0923 22:08:22.556305 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.556314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.556352 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.556801 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.556802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.556803 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:22.556842 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.556843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.557577 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.557578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.559180 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.559182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.559892 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.559892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.559950 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.559967 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.560010 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.560041 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154}
I20260923 22:08:22.560609 8360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 LEARNER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 22:08:22.560776 8312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 22:08:22.561355 8360 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.562343 8389 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f79a1786984d55a52d7e6a68f85969. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.562424 8389 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:22.562363 8498 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: NON_VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: true } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.562456 8498 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:22.562638 8311 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: attempting to promote NON_VOTER dd770ae25cd24f2882d85d8e09b4f69e to VOTER
I20260923 22:08:22.562815 8311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } }
I20260923 22:08:22.563213 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.563117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59494)
0923 22:08:22.563140 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.563192 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
I20260923 22:08:22.563261 8360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 LEARNER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 22:08:22.563452 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.563386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59506)
0923 22:08:22.563416 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.563434 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:39677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:08:22.563452 8499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260923 22:08:22.563545 7736 consensus_peers.cc:487] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 -> Peer 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188:39677): Couldn't send request to peer 95de76c2e6fc401da5ffa317182adc13. Status: Network error: Client connection negotiation failed: client connection to 127.1.161.188:39677: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:22.563962 8311 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, dd770ae25cd24f2882d85d8e09b4f69e (127.1.161.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.564117 8360 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, dd770ae25cd24f2882d85d8e09b4f69e (127.1.161.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.564695 8311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.564723 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.564764 8311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:22.564772 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:22.564875 8498 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.564945 8498 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:22.571885 8404 master.cc:552] Master@127.1.161.188:39677 shutdown complete.
I20260923 22:08:22.571950 8404 master_runner.cc:305] Clearing existing system tablet
I20260923 22:08:22.572067 8404 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.572125 8404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.572685 8404 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:22.573125 8502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.573228 8404 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.573275 8404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.573297 8404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "95de76c2e6fc401da5ffa317182adc13"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "5c736edd4c0961366ef8f02f0107e94d"
server_key_iv: "a9826c033515b73b52c5e091e9bb4720"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.573810 8404 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:22.575085 8404 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 22:08:22.575137 8404 log.cc:1196] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 22:08:22.575412 8404 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Deleting consensus metadata
I20260923 22:08:22.575536 8404 master_runner.cc:315] Copying system tablet from 127.1.161.190:38131
I20260923 22:08:22.576092 8404 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: tablet copy: Beginning tablet copy session from remote peer at address 127.1.161.190:38131
I20260923 22:08:22.581398 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.576403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.1.161.190:38131)
0923 22:08:22.576470 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.576472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.576537 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:08:22.576602 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.576602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.576611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.576661 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.576878 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.577520 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.578134 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.578397 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.578460 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.579337 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.579338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.579338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.579343 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.580517 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.580854 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.580854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.580855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.580898 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.581065 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.581066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.581066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.581144 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.581193 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.581311 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.581355 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:08:22.581459 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.576383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:45622)
0923 22:08:22.576475 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.576484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.576537 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.576663 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.576663 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.576777 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.576821 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.576822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.577590 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.577590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.578057 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.578058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.578465 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.578466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.578505 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.578931 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.578935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.579263 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.580595 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.580596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.580596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.580789 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.580958 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.580958 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.580959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.581012 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.581199 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.581200 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.581236 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.581299 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.581379 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":62,"threads_started":1}
I20260923 22:08:22.581729 7772 tablet_copy_service.cc:140] P 58f79a1786984d55a52d7e6a68f85969: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 95de76c2e6fc401da5ffa317182adc13 ({username='oryx', principal='oryx/127.1.161.188@KRBTEST.COM'} at 127.0.0.1:45622)
I20260923 22:08:22.581776 7772 tablet_copy_service.cc:161] P 58f79a1786984d55a52d7e6a68f85969: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 95de76c2e6fc401da5ffa317182adc13 at {username='oryx', principal='oryx/127.1.161.188@KRBTEST.COM'} at 127.0.0.1:45622: session id = 95de76c2e6fc401da5ffa317182adc13-00000000000000000000000000000000
I20260923 22:08:22.582132 7772 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 22:08:22.582504 8404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 22:08:22.583251 8404 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: tablet copy: Starting download of 0 data blocks...
I20260923 22:08:22.583376 8404 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: tablet copy: Starting download of 1 WAL segments...
I20260923 22:08:22.584014 8404 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 22:08:22.584662 7772 tablet_copy_service.cc:342] P 58f79a1786984d55a52d7e6a68f85969: Request end of tablet copy session 95de76c2e6fc401da5ffa317182adc13-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.161.188@KRBTEST.COM'} at 127.0.0.1:45622
I20260923 22:08:22.584725 7772 tablet_copy_service.cc:433] P 58f79a1786984d55a52d7e6a68f85969: ending tablet copy session 95de76c2e6fc401da5ffa317182adc13-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 95de76c2e6fc401da5ffa317182adc13
I20260923 22:08:22.586769 8404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.587813 8509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.587886 8510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.588027 8404 server_base.cc:1034] running on GCE node
W20260923 22:08:22.587900 8512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.588173 8404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.588331 8404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.589445 8404 hybrid_clock.cc:648] HybridClock initialized: now 1790201302589428 us; error 31 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.590641 8404 init.cc:377] Logged in from keytab as oryx/127.1.161.188@KRBTEST.COM (short username oryx)
I20260923 22:08:22.590942 8404 webserver.cc:460] Webserver started at http://127.1.161.188:34969/ using document root <none> and password file <none>
I20260923 22:08:22.591049 8404 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.591099 8404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.591697 8404 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.592201 8519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.592305 8404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:22.592344 8404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "95de76c2e6fc401da5ffa317182adc13"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "5c736edd4c0961366ef8f02f0107e94d"
server_key_iv: "a9826c033515b73b52c5e091e9bb4720"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.592386 8404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.597287 8404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.598654 8404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.601209 8404 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.188:39677
I20260923 22:08:22.601269 8571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.188:39677 every 8 connection(s)
I20260923 22:08:22.601450 8404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
I20260923 22:08:22.601946 8572 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:22.602281 8572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Bootstrap starting.
I20260923 22:08:22.604178 8572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:22.604390 8572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: Bootstrap complete.
I20260923 22:08:22.604548 8572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } }
I20260923 22:08:22.604602 8572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 95de76c2e6fc401da5ffa317182adc13, State: Initialized, Role: LEARNER
I20260923 22:08:22.604624 8572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } }
I20260923 22:08:22.604781 8572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:22.604791 8575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:22.604863 8575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:22.606108 8586 catalog_manager.cc:1259] Loaded cluster ID: 582e90cbf8a14b54ad7a7a6f449e80b7
I20260923 22:08:22.606151 8586 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: loading cluster ID for follower catalog manager: success
I20260923 22:08:22.607282 8586 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: acquiring CA information for follower catalog manager: success
I20260923 22:08:22.607515 8586 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:08:22.616169 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.610635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.1.161.188:39677)
0923 22:08:22.610754 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.610756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.610761 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:08:22.610775 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.610776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.610782 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.610846 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.611321 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.611849 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.612470 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.612643 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.612724 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.613711 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.613712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.613712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.613717 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.615002 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.615417 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.615418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.615419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.615473 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.615675 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.615676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.615676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.615785 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.615886 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.615995 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.616068 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260923 22:08:22.616637 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7814
I20260923 22:08:22.621574 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:33689
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:38131
--webserver_port=46523
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:22.695923 8588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.696126 8588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.696163 8588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.697139 8588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.697207 8588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.697244 8588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:22.698547 8588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:33689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=46523
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:22.698772 8588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.698968 8588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.703166 8594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.703229 8588 server_base.cc:1034] running on GCE node
W20260923 22:08:22.703224 8593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.703220 8596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.703465 8588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.703682 8588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.704826 8588 hybrid_clock.cc:648] HybridClock initialized: now 1790201302704811 us; error 26 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.707315 8588 init.cc:377] Logged in from keytab as oryx/127.1.161.129@KRBTEST.COM (short username oryx)
I20260923 22:08:22.707612 8588 webserver.cc:460] Webserver started at http://127.1.161.129:46523/ using document root <none> and password file <none>
I20260923 22:08:22.707760 8588 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.707803 8588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.708849 8588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:22.709306 8603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.709444 8588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.709515 8588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "6c49f2022bdf46ea82a080009276e4c5"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "48118adbfa9b923a966503c5ced75543"
server_key_iv: "fadc2fb20f2b09d43df70da8086a519b"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.709762 8588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.728297 8588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.729625 8588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.729715 8588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.729838 8588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:22.730089 8588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:22.730121 8588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.730139 8588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:22.730183 8588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.735661 8588 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:33689
I20260923 22:08:22.735710 8716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:33689 every 8 connection(s)
I20260923 22:08:22.736068 8588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
I20260923 22:08:22.745460 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.737042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:34549 (local address 127.1.161.190:38131)
0923 22:08:22.737084 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.737086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.737283 (+ 197us) server_negotiation.cc:407] Connection header received
0923 22:08:22.737570 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.737571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.737582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.737620 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.738188 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738863 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.739673 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.739920 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.739976 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.742184 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.742185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.742186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.742192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.743515 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744479 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.744481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.744481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.744529 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744816 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.744817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.744817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.744930 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.745000 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.745294 (+ 294us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.745372 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:22.745476 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.736686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:50993 (local address 127.1.161.189:45695)
0923 22:08:22.736724 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.736726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.737192 (+ 466us) server_negotiation.cc:407] Connection header received
0923 22:08:22.737556 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.737558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.737570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.737598 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.738226 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738829 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.740083 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.740248 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.740325 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.743067 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.743068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.743069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.743075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.744271 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744624 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.744625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.744625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.744675 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744864 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.744865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.744866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.744997 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.745062 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.745303 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.745384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:22.745594 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.736984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:34549)
0923 22:08:22.737235 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.737241 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.737471 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.737677 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.737681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.737929 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.738069 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.738072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738920 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.738921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.739592 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.739594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.739989 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.739989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.740123 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.740633 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.740637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.742101 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.743932 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.743938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.743940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.744405 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.744599 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.744600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.744715 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.744993 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.744998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.745125 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.745249 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.745392 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":23,"thread_start_us":64,"threads_started":1}
I20260923 22:08:22.745759 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.736725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.1.161.129:50993)
0923 22:08:22.737141 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.737148 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.737454 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.737991 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.737992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.738114 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.738160 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.738161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738910 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.738911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.739672 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.739674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.740614 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.740615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.740656 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.740993 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.740997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.742966 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.744351 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.744352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.744547 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.744741 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.744742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.744789 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.745049 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.745049 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.745125 (+ 76us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.745576 (+ 451us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.745668 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":25,"thread_start_us":127,"threads_started":1}
I20260923 22:08:22.745730 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8588
I20260923 22:08:22.745903 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7948
I20260923 22:08:22.746179 8719 heartbeater.cc:347] Connected to a master server at 127.1.161.189:45695
I20260923 22:08:22.746258 8719 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.746426 8719 heartbeater.cc:511] Master 127.1.161.189:45695 requested a full tablet report, sending...
I20260923 22:08:22.746771 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.736710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:48567 (local address 127.1.161.188:39677)
0923 22:08:22.736781 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.736783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.736955 (+ 172us) server_negotiation.cc:407] Connection header received
0923 22:08:22.737557 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.737558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.737570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.737598 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.738164 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738609 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.739714 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.739983 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.740043 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.743368 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.743369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.743369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.743375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.744784 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.745077 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.745078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.745078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.745125 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.745362 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.745363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.745363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.745479 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.745523 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.746586 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.746654 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:22.746802 8350 ts_manager.cc:194] Registered new tserver with Master: 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:22.747056 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.736708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.1.161.129:48567)
0923 22:08:22.736857 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.736890 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.737452 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.737678 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.737681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.737964 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.738070 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.738075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.738692 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.738693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.739616 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.739618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.740048 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.740049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.740123 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.740582 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.740608 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.743258 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.744848 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.744848 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.744848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.745009 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.745202 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.745202 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.745202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.745287 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.746026 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.746027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.746069 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.746839 (+ 770us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.746927 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260923 22:08:22.747962 8717 heartbeater.cc:347] Connected to a master server at 127.1.161.188:39677
I20260923 22:08:22.748006 8717 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.748078 8717 heartbeater.cc:511] Master 127.1.161.188:39677 requested a full tablet report, sending...
I20260923 22:08:22.748373 8536 ts_manager.cc:194] Registered new tserver with Master: 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:22.753669 8718 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:22.753731 8718 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.753731 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:44831
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:38131
--webserver_port=34581
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:22.753791 8718 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
I20260923 22:08:22.754047 7752 ts_manager.cc:194] Re-registered known tserver with Master: 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:22.754519 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:34549
W20260923 22:08:22.822971 8724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.823122 8724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.823140 8724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.824261 8724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.824309 8724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.824368 8724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:22.825752 8724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:44831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=34581
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:22.825978 8724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.826201 8724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:22.830725 8729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.830700 8730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.830767 8732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.830899 8724 server_base.cc:1034] running on GCE node
I20260923 22:08:22.831048 8724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.831256 8724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.832394 8724 hybrid_clock.cc:648] HybridClock initialized: now 1790201302832386 us; error 23 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.834730 8724 init.cc:377] Logged in from keytab as oryx/127.1.161.130@KRBTEST.COM (short username oryx)
I20260923 22:08:22.835058 8724 webserver.cc:460] Webserver started at http://127.1.161.130:34581/ using document root <none> and password file <none>
I20260923 22:08:22.835314 8724 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.835368 8724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.836534 8724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:22.836970 8739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.837118 8724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.837200 8724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "56fe4644f0f84222a0ac1d6b641cb2ab"
format_stamp: "Formatted at 2026-09-23 22:08:21 on dist-test-slave-npvd"
server_key: "8e3ae02b0216844434637e6cde8b6f9c"
server_key_iv: "aa840df3d1aaed4361ac0f2484216f36"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.837463 8724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.848012 8724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.849486 8724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.849618 8724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.849798 8724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:22.850045 8724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:22.850095 8724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.850127 8724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:22.850148 8724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.855058 8724 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:44831
I20260923 22:08:22.855089 8852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:44831 every 8 connection(s)
I20260923 22:08:22.855412 8724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:22.857954 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8724
I20260923 22:08:22.858096 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8082
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
I20260923 22:08:22.863824 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:46331
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:38131
--webserver_port=33333
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:22.866766 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:43131 (local address 127.1.161.190:38131)
0923 22:08:22.856360 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.856361 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.856500 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:08:22.857473 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.857474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.857485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.857525 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.858698 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.859380 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860215 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.860452 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860502 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.863392 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.863393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.863394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.863402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.864841 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865591 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.865592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.865593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.865654 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.866114 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.866115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.866115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.866231 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.866311 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.866594 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.866669 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:08:22.866925 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:43131)
0923 22:08:22.856383 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.856418 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.857322 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.857564 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.857564 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.857922 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.858134 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.858625 (+ 491us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.859472 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.859473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.860150 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.860152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.860571 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.860626 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.861388 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.861392 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.863267 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.865116 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.865124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.865506 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.865970 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.865971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.866032 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.866303 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.866309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.866446 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.866570 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.866716 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":21,"spinlock_wait_cycles":6016,"thread_start_us":92,"threads_started":1}
I20260923 22:08:22.867324 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:46171 (local address 127.1.161.189:45695)
0923 22:08:22.856272 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.856274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.856502 (+ 228us) server_negotiation.cc:407] Connection header received
0923 22:08:22.857415 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.857416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.857427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.857479 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.858218 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.858778 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860057 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.860315 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860349 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.863748 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.863749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.863750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.863756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.865171 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865622 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.865623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.865624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.865676 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865945 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.865947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.865947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.866095 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.866166 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.867100 (+ 934us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.867191 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260923 22:08:22.867355 8854 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:22.867426 8854 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.867579 8854 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
I20260923 22:08:22.867697 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.1.161.130:46171)
0923 22:08:22.856366 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.856418 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.857322 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.857531 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.857534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.857921 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.858134 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.858139 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.858935 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.858935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.859979 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.859981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.860416 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.860524 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.861318 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.861326 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.863656 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.865291 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.865293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.865537 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.865766 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.865767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.865767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.865886 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.866947 (+ 1061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.866948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.866989 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.867422 (+ 433us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.867522 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":7,"thread_start_us":97,"threads_started":1}
I20260923 22:08:22.867949 7752 ts_manager.cc:194] Re-registered known tserver with Master: 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831)
I20260923 22:08:22.868006 8855 heartbeater.cc:347] Connected to a master server at 127.1.161.189:45695
I20260923 22:08:22.868042 8855 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.868101 8855 heartbeater.cc:511] Master 127.1.161.189:45695 requested a full tablet report, sending...
I20260923 22:08:22.868211 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:38873 (local address 127.1.161.188:39677)
0923 22:08:22.856266 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.856269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.856632 (+ 363us) server_negotiation.cc:407] Connection header received
0923 22:08:22.857443 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.857445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.857454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.857481 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.859178 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.859865 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.860882 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.861161 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.861207 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.864952 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.864954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.864954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.864960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.866865 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.867475 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.867477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.867477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.867530 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.867714 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.867715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.867715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.867851 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.867901 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.868069 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.868136 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260923 22:08:22.868340 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:43131
I20260923 22:08:22.868351 8350 ts_manager.cc:194] Registered new tserver with Master: 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831)
I20260923 22:08:22.868623 8858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.856050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.1.161.130:38873)
0923 22:08:22.856588 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.856596 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.857335 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.857549 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.857550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.857900 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.858157 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.858160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.859987 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.859988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.860819 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.860821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.861278 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.861279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.861344 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.861679 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.861683 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.864199 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.867110 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.867111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.867111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.867396 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.867597 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.867598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.867598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.867649 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.867943 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.867943 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.867975 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.868308 (+ 333us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.868386 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260923 22:08:22.868904 8853 heartbeater.cc:347] Connected to a master server at 127.1.161.188:39677
I20260923 22:08:22.868953 8853 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.869010 8853 heartbeater.cc:511] Master 127.1.161.188:39677 requested a full tablet report, sending...
I20260923 22:08:22.869271 8536 ts_manager.cc:194] Registered new tserver with Master: 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831)
W20260923 22:08:22.939021 8860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:22.939200 8860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:22.939220 8860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:22.940263 8860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:22.940299 8860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:22.940322 8860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:22.941694 8860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:46331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=33333
--tserver_master_addrs=127.1.161.190:38131,127.1.161.189:45695,127.1.161.188:39677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:22.941856 8860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:22.942010 8860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:22.946403 8860 server_base.cc:1034] running on GCE node
W20260923 22:08:22.946364 8865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.946388 8868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:22.946373 8866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:22.946820 8860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:22.947023 8860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:22.948180 8860 hybrid_clock.cc:648] HybridClock initialized: now 1790201302948170 us; error 51 us; skew 500 ppm
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:22.950399 8860 init.cc:377] Logged in from keytab as oryx/127.1.161.131@KRBTEST.COM (short username oryx)
I20260923 22:08:22.950765 8860 webserver.cc:460] Webserver started at http://127.1.161.131:33333/ using document root <none> and password file <none>
I20260923 22:08:22.950911 8860 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:22.950954 8860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:22.952044 8860 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:22.952489 8875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.952644 8860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.952728 8860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b7f6337539c245fbae3be625407f769a"
format_stamp: "Formatted at 2026-09-23 22:08:22 on dist-test-slave-npvd"
server_key: "19f36a4be865eeb0e193a5d76b7691c7"
server_key_iv: "65b54f78de4029252e88cedce9793d04"
server_key_version: "encryptionkey@0"
I20260923 22:08:22.952992 8860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:22.974280 8860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:22.976437 8860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:22.976573 8860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:22.976754 8860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:22.976999 8860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:22.977048 8860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.977080 8860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:22.977102 8860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:22.982059 8860 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:46331
I20260923 22:08:22.982106 8988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:46331 every 8 connection(s)
I20260923 22:08:22.982412 8860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.188@KRBTEST.COM
Sep 23 22:08:22 dist-test-slave-npvd krb5kdc[7707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201302, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.189@KRBTEST.COM
I20260923 22:08:22.989306 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 8860
I20260923 22:08:22.992578 8994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.982993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:33415)
0923 22:08:22.983433 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.983441 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.983679 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.983907 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.983908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.984199 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.984310 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.984317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.985129 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.985130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.986188 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.986190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986636 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.986696 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.987127 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.987132 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.989014 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.991266 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991266 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.991267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.991464 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.991641 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.991642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.991750 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.992044 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.992048 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.992179 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.992267 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.992405 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":104,"thread_start_us":83,"threads_started":1}
I20260923 22:08:22.992703 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.982992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45695 (local address 127.1.161.131:50041)
0923 22:08:22.983142 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.983193 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.983656 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.983870 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.983873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.984152 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.984309 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.984312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.985301 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.985302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.986188 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.986190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986724 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.986798 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.987330 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.987335 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.989281 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.990853 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.990857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.990859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.991040 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.992019 (+ 979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.992019 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.992020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.992150 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.992433 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.992434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.992471 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.992525 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.992615 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260923 22:08:22.992956 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.983322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:33415 (local address 127.1.161.190:38131)
0923 22:08:22.983368 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.983370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.983482 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:08:22.983785 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.983786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.983798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.983826 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.984426 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.985050 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986279 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986558 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986624 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.989096 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.989097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.989098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.989105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.991184 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991532 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.991533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.991534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.991582 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991817 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.991818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.991819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.991939 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.991995 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.992795 (+ 800us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.992880 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:22.993036 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.983272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:50041 (local address 127.1.161.189:45695)
0923 22:08:22.983353 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.983355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.983362 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:22.983748 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.983749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.983759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.983816 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.984397 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.985050 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986317 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986605 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986671 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.989396 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.989397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.989398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.989404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.990770 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991131 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.991133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.991133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.991198 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.992213 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.992215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.992215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.992350 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.992408 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.992881 (+ 473us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.992960 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 22:08:22.993315 8991 heartbeater.cc:347] Connected to a master server at 127.1.161.189:45695
I20260923 22:08:22.993387 8991 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.993525 8991 heartbeater.cc:511] Master 127.1.161.189:45695 requested a full tablet report, sending...
I20260923 22:08:22.993670 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.983269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:45133 (local address 127.1.161.188:39677)
0923 22:08:22.983317 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.983319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.983465 (+ 146us) server_negotiation.cc:407] Connection header received
0923 22:08:22.983771 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.983772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.983784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.983816 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.984494 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.984917 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986093 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986360 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986434 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.988877 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.988879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.988879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.988886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.991408 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991895 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.991896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.991896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.991950 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.993210 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.993210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.993211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.993328 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.993364 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.993532 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.993594 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:22.993695 8990 heartbeater.cc:347] Connected to a master server at 127.1.161.190:38131
I20260923 22:08:22.993726 8990 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.993781 8990 heartbeater.cc:511] Master 127.1.161.190:38131 requested a full tablet report, sending...
I20260923 22:08:22.993908 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.982993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.1.161.131:45133)
0923 22:08:22.983404 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:22.983414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:22.983680 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:22.984220 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:22.984221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:22.984354 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:22.984405 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.984406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.985028 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.985029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.985994 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:22.985996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.986458 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.986459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:22.986581 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.986998 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:22.987005 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:22.988767 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:22.991586 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.991587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.991588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.991797 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.992015 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:22.992016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:22.992016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:22.992077 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:22.993406 (+ 1329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:22.993407 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:22.993439 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:08:22.993740 (+ 301us) client_negotiation.cc:240] Negotiation successful
0923 22:08:22.993823 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":87,"spinlock_wait_cycles":5376,"thread_start_us":85,"threads_started":1}
I20260923 22:08:22.994050 8350 ts_manager.cc:194] Registered new tserver with Master: b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:22.994170 8989 heartbeater.cc:347] Connected to a master server at 127.1.161.188:39677
I20260923 22:08:22.994215 8989 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:22.994248 7752 ts_manager.cc:194] Re-registered known tserver with Master: b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:22.994264 8989 heartbeater.cc:511] Master 127.1.161.188:39677 requested a full tablet report, sending...
I20260923 22:08:22.994527 8536 ts_manager.cc:194] Registered new tserver with Master: b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:22.994737 7752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:33415
I20260923 22:08:22.999752 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.990354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.1.161.190:38131)
0923 22:08:22.991291 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.991293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.991387 (+ 94us) server_negotiation.cc:407] Connection header received
0923 22:08:22.991468 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.991469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.991479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.991519 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.992223 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.993492 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.994987 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.995298 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.995341 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.996820 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.996821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.996822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.996827 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.998282 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.998864 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.998865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.998865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.998912 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.999078 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.999079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.999079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.999248 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.999334 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.999580 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.999650 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":72,"threads_started":1}
I20260923 22:08:22.999815 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.990746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.1.161.188:39677)
0923 22:08:22.991233 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.991235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.991684 (+ 449us) server_negotiation.cc:407] Connection header received
0923 22:08:22.991802 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.991803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.991813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.991849 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.992513 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.993138 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.994969 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.995316 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.995377 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.997198 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.997199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.997200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.997206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:22.998397 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.998793 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.998794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.998795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.998842 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:22.999037 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:22.999039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:22.999039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:22.999145 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:22.999268 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:22.999548 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 22:08:22.999703 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":50,"threads_started":1}
I20260923 22:08:23.001875 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:22.991516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.1.161.189:45695)
0923 22:08:22.993088 (+ 1572us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:22.993089 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:22.995054 (+ 1965us) server_negotiation.cc:407] Connection header received
0923 22:08:22.995133 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:22.995133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:22.995141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:22.995185 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:22.995918 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.996351 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.997283 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:22.998064 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:22.998129 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:22.999405 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:22.999406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:22.999407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:22.999412 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:23.000940 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.001251 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.001251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.001252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.001295 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.001465 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.001466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.001466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.001555 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:23.001636 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.001728 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.001789 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1561,"thread_start_us":39,"threads_started":1}
I20260923 22:08:23.001025 7752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:23.002487 7752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:23.009253 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.006197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.1.161.130:44831)
0923 22:08:23.006371 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.006376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.006385 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:23.006424 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.006427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.006453 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.006534 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.006643 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007264 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.008448 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.008945 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.009030 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.009085 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.009158 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.009187 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260923 22:08:23.009244 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.005795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:44831 (local address 127.0.0.1:45806)
0923 22:08:23.005894 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.005904 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.005940 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.006531 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.006532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.006534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.006589 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.006590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007358 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.007359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.008366 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.008367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.009029 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.009030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.009092 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.009096 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.009146 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.009180 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":23,"thread_start_us":63,"threads_started":1}
I20260923 22:08:23.009346 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.005744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:33689 (local address 127.0.0.1:55908)
0923 22:08:23.005875 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.005885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.005930 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.006300 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.006301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.006303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.006337 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.006338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007444 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.007445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.008045 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.008047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.009159 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.009160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.009210 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.009214 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.009239 (+ 25us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.009272 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260923 22:08:23.009883 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.005970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.1.161.129:33689)
0923 22:08:23.006144 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.006149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.006157 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:23.006192 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.006195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.006210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.006405 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.006472 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007367 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.008140 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.009074 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.009444 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.009728 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.009752 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.009811 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260923 22:08:23.009976 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.005704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46331 (local address 127.0.0.1:58120)
0923 22:08:23.005736 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.005747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.005790 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.006224 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.006225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.006227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.006265 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.006267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007389 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.007390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.008355 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.008357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.009808 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.009809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.009860 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.009862 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.009888 (+ 26us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.009922 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":32}
I20260923 22:08:23.009871 8787 tablet_service.cc:1467] Processing CreateTablet for tablet bc856a69330d4e2c9351aa79b885cf21 (DEFAULT_TABLE table=test-table [id=6a9098df589d47508b1b834bd3912f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:23.010180 8787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc856a69330d4e2c9351aa79b885cf21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:23.010411 8651 tablet_service.cc:1467] Processing CreateTablet for tablet bc856a69330d4e2c9351aa79b885cf21 (DEFAULT_TABLE table=test-table [id=6a9098df589d47508b1b834bd3912f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:23.010526 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.005936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.1.161.131:46331)
0923 22:08:23.006057 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.006061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.006070 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:23.006107 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.006109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.006123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.006347 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.006389 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.007303 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.008436 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.009727 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.010255 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.010328 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.010354 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.010376 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260923 22:08:23.010712 8651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc856a69330d4e2c9351aa79b885cf21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:23.011600 8923 tablet_service.cc:1467] Processing CreateTablet for tablet bc856a69330d4e2c9351aa79b885cf21 (DEFAULT_TABLE table=test-table [id=6a9098df589d47508b1b834bd3912f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:23.011850 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc856a69330d4e2c9351aa79b885cf21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:23.013270 9015 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Bootstrap starting.
I20260923 22:08:23.013628 9016 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Bootstrap starting.
I20260923 22:08:23.013914 9015 tablet_bootstrap.cc:654] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:23.014117 9016 tablet_bootstrap.cc:654] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:23.014189 9015 log.cc:824] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:23.014357 9017 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Bootstrap starting.
I20260923 22:08:23.014395 9016 log.cc:824] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:23.014981 9017 tablet_bootstrap.cc:654] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:23.015149 9016 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: No bootstrap required, opened a new log
I20260923 22:08:23.015240 9016 ts_tablet_manager.cc:1397] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:23.015308 9017 log.cc:824] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:23.015722 9015 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: No bootstrap required, opened a new log
I20260923 22:08:23.015785 9015 ts_tablet_manager.cc:1397] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:08:23.016000 9017 tablet_bootstrap.cc:492] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: No bootstrap required, opened a new log
I20260923 22:08:23.016069 9017 ts_tablet_manager.cc:1397] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:23.017153 9016 raft_consensus.cc:348] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.017292 9016 raft_consensus.cc:374] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:23.017324 9016 raft_consensus.cc:729] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c49f2022bdf46ea82a080009276e4c5, State: Initialized, Role: FOLLOWER
I20260923 22:08:23.017443 9016 consensus_queue.cc:260] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.017686 8718 heartbeater.cc:503] Master 127.1.161.190:38131 was elected leader, sending a full tablet report...
I20260923 22:08:23.017861 9017 raft_consensus.cc:348] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.017997 9017 raft_consensus.cc:374] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:23.018030 9017 raft_consensus.cc:729] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f6337539c245fbae3be625407f769a, State: Initialized, Role: FOLLOWER
I20260923 22:08:23.018133 9017 consensus_queue.cc:260] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.018365 9017 ts_tablet_manager.cc:1428] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:08:23.018472 8990 heartbeater.cc:503] Master 127.1.161.190:38131 was elected leader, sending a full tablet report...
I20260923 22:08:23.018844 9016 ts_tablet_manager.cc:1428] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260923 22:08:23.018884 9015 raft_consensus.cc:348] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.019013 9015 raft_consensus.cc:374] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:23.019049 9015 raft_consensus.cc:729] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fe4644f0f84222a0ac1d6b641cb2ab, State: Initialized, Role: FOLLOWER
I20260923 22:08:23.019189 9015 consensus_queue.cc:260] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.019412 9015 ts_tablet_manager.cc:1428] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 22:08:23.019670 8854 heartbeater.cc:503] Master 127.1.161.190:38131 was elected leader, sending a full tablet report...
I20260923 22:08:23.078866 9023 raft_consensus.cc:482] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:23.078935 9023 raft_consensus.cc:504] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.079221 9023 leader_election.cc:288] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7f6337539c245fbae3be625407f769a (127.1.161.131:46331), 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:23.081641 8858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.079335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46331 (local address 127.1.161.130:43943)
0923 22:08:23.079386 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.079397 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.079435 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.079623 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.079624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.079626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.079686 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.079687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080269 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.080270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.080986 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.080987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081461 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.081535 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.081537 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.081576 (+ 39us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.081600 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":24}
I20260923 22:08:23.081691 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.079413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:43943 (local address 127.1.161.131:46331)
0923 22:08:23.079467 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.079468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.079475 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:23.079495 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.079496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.079506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.079615 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.079760 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080186 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081076 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081386 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081434 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.081440 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.081596 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.081626 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:08:23.081776 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.079346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:33689 (local address 127.1.161.130:33007)
0923 22:08:23.079390 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.079399 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.079444 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.079633 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.079634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.079636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.079689 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.079690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080315 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.080316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.080986 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.080988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081515 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.081613 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.081616 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.081667 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.081699 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":3}
I20260923 22:08:23.081791 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.079408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:33007 (local address 127.1.161.129:33689)
0923 22:08:23.079442 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.079444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.079451 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:23.079528 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.079529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.079536 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.079607 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.079813 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.080205 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081077 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.081395 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.081479 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.081485 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.081691 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.081724 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:23.082156 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc856a69330d4e2c9351aa79b885cf21" candidate_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f6337539c245fbae3be625407f769a" is_pre_election: true
I20260923 22:08:23.082278 8943 raft_consensus.cc:2454] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56fe4644f0f84222a0ac1d6b641cb2ab in term 0.
I20260923 22:08:23.082235 8671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc856a69330d4e2c9351aa79b885cf21" candidate_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c49f2022bdf46ea82a080009276e4c5" is_pre_election: true
I20260923 22:08:23.082360 8671 raft_consensus.cc:2454] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56fe4644f0f84222a0ac1d6b641cb2ab in term 0.
I20260923 22:08:23.082456 8741 leader_election.cc:302] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fe4644f0f84222a0ac1d6b641cb2ab, b7f6337539c245fbae3be625407f769a; no voters:
I20260923 22:08:23.082556 9023 raft_consensus.cc:2781] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:23.082593 9023 raft_consensus.cc:482] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:23.082615 9023 raft_consensus.cc:3037] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:23.083141 9023 raft_consensus.cc:504] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.083321 9023 leader_election.cc:288] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [CANDIDATE]: Term 1 election: Requested vote from peers b7f6337539c245fbae3be625407f769a (127.1.161.131:46331), 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689)
I20260923 22:08:23.083419 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc856a69330d4e2c9351aa79b885cf21" candidate_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f6337539c245fbae3be625407f769a"
I20260923 22:08:23.083479 8943 raft_consensus.cc:3037] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:23.083451 8671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc856a69330d4e2c9351aa79b885cf21" candidate_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c49f2022bdf46ea82a080009276e4c5"
I20260923 22:08:23.083530 8671 raft_consensus.cc:3037] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:23.084024 8943 raft_consensus.cc:2454] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56fe4644f0f84222a0ac1d6b641cb2ab in term 1.
I20260923 22:08:23.084209 8741 leader_election.cc:302] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fe4644f0f84222a0ac1d6b641cb2ab, b7f6337539c245fbae3be625407f769a; no voters:
I20260923 22:08:23.084335 9023 raft_consensus.cc:2781] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:23.084465 9023 raft_consensus.cc:686] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 LEADER]: Becoming Leader. State: Replica: 56fe4644f0f84222a0ac1d6b641cb2ab, State: Running, Role: LEADER
I20260923 22:08:23.084470 8671 raft_consensus.cc:2454] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56fe4644f0f84222a0ac1d6b641cb2ab in term 1.
I20260923 22:08:23.084542 9023 consensus_queue.cc:237] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } }
I20260923 22:08:23.085489 7752 catalog_manager.cc:5495] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130). New cstate: current_term: 1 leader_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:23.100600 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.098564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.1.161.130:44831)
0923 22:08:23.098616 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.098618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.098625 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:23.098678 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.098678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.098691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.098775 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:23.098878 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.099328 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.099940 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.100113 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.100180 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.100334 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:23.100415 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:23.100488 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.100530 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.100545 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:23.101917 8671 raft_consensus.cc:1261] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Refusing update from remote peer 56fe4644f0f84222a0ac1d6b641cb2ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:23.101917 8943 raft_consensus.cc:1261] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Refusing update from remote peer 56fe4644f0f84222a0ac1d6b641cb2ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:23.102191 9024 consensus_queue.cc:1035] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:23.102276 9023 consensus_queue.cc:1035] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:23.104918 9029 mvcc.cc:205] Tried to move back new op lower bound from 7332664537503141888 to 7332664537434669056. Current Snapshot: MvccSnapshot[applied={T|T < 7332664537503141888}]
I20260923 22:08:23.105512 9028 mvcc.cc:205] Tried to move back new op lower bound from 7332664537503141888 to 7332664537434669056. Current Snapshot: MvccSnapshot[applied={T|T < 7332664537503141888}]
I20260923 22:08:23.105675 9027 mvcc.cc:205] Tried to move back new op lower bound from 7332664537503141888 to 7332664537434669056. Current Snapshot: MvccSnapshot[applied={T|T < 7332664537503141888}]
I20260923 22:08:23.107892 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.104511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.1.161.188:39677)
0923 22:08:23.104558 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.104559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.104566 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:23.104613 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.104614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.104631 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.104680 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:23.104830 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.105523 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.107053 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.107528 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.107584 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.107640 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.107812 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.107844 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:23.107887 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.104482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59530)
0923 22:08:23.104519 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:23.104527 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:23.104561 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:23.104716 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:23.104717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:23.104718 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:23.104764 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.104766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.106020 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.106021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.107000 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:23.107003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.107669 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.107669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:23.107720 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.107723 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:23.107798 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:08:23.107831 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:08:23.108384 8546 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260923 22:08:23.108439 7752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:23.108498 7752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:23.109308 9037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f79a1786984d55a52d7e6a68f85969. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:23.109386 8500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58f79a1786984d55a52d7e6a68f85969" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } }
I20260923 22:08:23.109473 8500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:23.109678 7752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: Sending DeleteTablet for 3 replicas of tablet bc856a69330d4e2c9351aa79b885cf21
I20260923 22:08:23.109944 8651 tablet_service.cc:1514] Processing DeleteTablet for tablet bc856a69330d4e2c9351aa79b885cf21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:23 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:55908
I20260923 22:08:23.109983 8500 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969: attempting to promote NON_VOTER 95de76c2e6fc401da5ffa317182adc13 to VOTER
I20260923 22:08:23.110092 8500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } }
I20260923 22:08:23.110456 8923 tablet_service.cc:1514] Processing DeleteTablet for tablet bc856a69330d4e2c9351aa79b885cf21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:23 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:58120
I20260923 22:08:23.110631 9043 tablet_replica.cc:330] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: stopping tablet replica
I20260923 22:08:23.110635 8787 tablet_service.cc:1514] Processing DeleteTablet for tablet bc856a69330d4e2c9351aa79b885cf21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:23 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:45806
I20260923 22:08:23.110733 9043 raft_consensus.cc:2229] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:23.110831 9043 raft_consensus.cc:2258] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:23.110832 9044 tablet_replica.cc:330] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: stopping tablet replica
I20260923 22:08:23.110889 8360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260923 22:08:23.111035 9044 raft_consensus.cc:2229] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:23.111117 8499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58f79a1786984d55a52d7e6a68f85969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260923 22:08:23.111156 9044 raft_consensus.cc:2258] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:23.111255 9043 ts_tablet_manager.cc:1905] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:23.111394 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 7720
I20260923 22:08:23.111619 9044 ts_tablet_manager.cc:1905] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:23.110431 8545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Refusing update from remote peer 58f79a1786984d55a52d7e6a68f85969: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260923 22:08:23.113214 9037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: LEARNER
I20260923 22:08:23.113327 9042 tablet_replica.cc:330] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: stopping tablet replica
I20260923 22:08:23.113404 9042 raft_consensus.cc:2229] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:23.113483 9042 raft_consensus.cc:2258] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:23.113658 9044 ts_tablet_manager.cc:1918] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:23.113723 9044 log.cc:1196] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/bc856a69330d4e2c9351aa79b885cf21
I20260923 22:08:23.113973 9042 ts_tablet_manager.cc:1905] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:23.114042 9044 ts_tablet_manager.cc:1939] T bc856a69330d4e2c9351aa79b885cf21 P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting consensus metadata
I20260923 22:08:23.115393 9042 ts_tablet_manager.cc:1918] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:23.115468 9042 log.cc:1196] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/bc856a69330d4e2c9351aa79b885cf21
I20260923 22:08:23.115718 9042 ts_tablet_manager.cc:1939] T bc856a69330d4e2c9351aa79b885cf21 P 6c49f2022bdf46ea82a080009276e4c5: Deleting consensus metadata
W20260923 22:08:23.118957 8522 connection.cc:535] server connection from 127.0.0.1:59530 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 22:08:23.118984 8741 connection.cc:535] server connection from 127.0.0.1:45806 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 22:08:23.119127 8337 connection.cc:535] server connection from 127.0.0.1:46456 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 22:08:23.119253 8605 connection.cc:535] server connection from 127.0.0.1:55908 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:08:23.121774 9043 ts_tablet_manager.cc:1918] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:23.121847 9043 log.cc:1196] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/bc856a69330d4e2c9351aa79b885cf21
I20260923 22:08:23.122052 9043 ts_tablet_manager.cc:1939] T bc856a69330d4e2c9351aa79b885cf21 P b7f6337539c245fbae3be625407f769a: Deleting consensus metadata
W20260923 22:08:23.122332 8876 connection.cc:441] server connection from 127.0.0.1:58120 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58120 (request call id 1) could send its response
I20260923 22:08:23.127151 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.120583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.1.161.189:45695)
0923 22:08:23.120613 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.120615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.120722 (+ 107us) server_negotiation.cc:407] Connection header received
0923 22:08:23.120742 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.120743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.120752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.120798 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:23.121429 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.121976 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.122738 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.122993 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.123054 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.124421 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:23.124423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:23.124423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:23.124428 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:23.126089 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.126491 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.126492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.126492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.126552 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.126737 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.126738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.126738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.126839 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:23.126933 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.127013 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.127075 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:23.127353 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:23.120493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.1.161.188:39677)
0923 22:08:23.120524 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:23.120526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:23.120850 (+ 324us) server_negotiation.cc:407] Connection header received
0923 22:08:23.120870 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:23.120871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:23.120879 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:23.120923 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:23.121431 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.122041 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.123327 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:23.123570 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:23.123624 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:23.124670 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:23.124671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:23.124672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:23.124677 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:23.126182 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.126616 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.126618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.126618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.126659 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:23.126854 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:23.126854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:23.126855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:23.126961 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:23.127018 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:23.127201 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 22:08:23.127281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:24.105412 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.105165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:54479)
0923 22:08:24.105281 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.105355 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
W20260923 22:08:24.105667 8854 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:38131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.105871 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.105433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:54453)
0923 22:08:24.105677 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.105834 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":42,"threads_started":1}
W20260923 22:08:24.106033 8990 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:38131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.106612 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.106368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:42139)
0923 22:08:24.106500 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.106567 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
W20260923 22:08:24.106828 8718 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:38131 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.614105 9057 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 58f79a1786984d55a52d7e6a68f85969)
I20260923 22:08:24.614195 9057 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } }
I20260923 22:08:24.614480 9057 leader_election.cc:288] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58f79a1786984d55a52d7e6a68f85969 (127.1.161.190:38131), 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188:39677)
I20260923 22:08:24.614774 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.614568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46088)
0923 22:08:24.614650 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.614730 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67,"thread_start_us":49,"threads_started":1}
W20260923 22:08:24.614886 8337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.615243 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.615120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46102)
0923 22:08:24.615186 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.615220 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:08:24.615335 8337 leader_election.cc:334] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 58f79a1786984d55a52d7e6a68f85969 (127.1.161.190:38131): Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.617447 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.614589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:39677 (local address 127.0.0.1:59546)
0923 22:08:24.614688 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.614696 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.614763 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.614973 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.614974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.614977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.615029 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.615031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.615829 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.615830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.616604 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.616606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.617194 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.617195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.617254 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.617258 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.617338 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.617378 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":41,"threads_started":1}
I20260923 22:08:24.617436 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.614765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.1.161.188:39677)
0923 22:08:24.614865 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.614867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.614875 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:24.614899 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.614900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.614908 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.614971 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.615078 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.615742 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.616705 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.617100 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.617191 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.617202 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.617345 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.617377 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":39,"threads_started":1}
I20260923 22:08:24.617709 8546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "95de76c2e6fc401da5ffa317182adc13" is_pre_election: true
I20260923 22:08:24.617797 8546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate dd770ae25cd24f2882d85d8e09b4f69e in term 1.
I20260923 22:08:24.617949 8334 leader_election.cc:302] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95de76c2e6fc401da5ffa317182adc13, dd770ae25cd24f2882d85d8e09b4f69e; no voters: 58f79a1786984d55a52d7e6a68f85969
I20260923 22:08:24.618070 9057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 22:08:24.618111 9057 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 58f79a1786984d55a52d7e6a68f85969)
I20260923 22:08:24.618136 9057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:08:24.618826 9057 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } }
I20260923 22:08:24.618957 9057 leader_election.cc:288] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 election: Requested vote from peers 58f79a1786984d55a52d7e6a68f85969 (127.1.161.190:38131), 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188:39677)
I20260923 22:08:24.619069 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.618997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46114)
0923 22:08:24.619026 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.619047 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 22:08:24.619072 8546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "95de76c2e6fc401da5ffa317182adc13"
I20260923 22:08:24.619145 8546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 1 LEARNER]: Advancing to term 2
I20260923 22:08:24.619418 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.619322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46124)
0923 22:08:24.619372 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.619399 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 22:08:24.619499 8337 leader_election.cc:334] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 58f79a1786984d55a52d7e6a68f85969 (127.1.161.190:38131): Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
I20260923 22:08:24.619851 8546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 2 LEARNER]: Leader election vote request: Granting yes vote for candidate dd770ae25cd24f2882d85d8e09b4f69e in term 2.
I20260923 22:08:24.620004 8334 leader_election.cc:302] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95de76c2e6fc401da5ffa317182adc13, dd770ae25cd24f2882d85d8e09b4f69e; no voters: 58f79a1786984d55a52d7e6a68f85969
I20260923 22:08:24.620093 9057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:08:24.620194 9057 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 2 LEADER]: Becoming Leader. State: Replica: dd770ae25cd24f2882d85d8e09b4f69e, State: Running, Role: LEADER
I20260923 22:08:24.620296 9057 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } }
I20260923 22:08:24.620592 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd770ae25cd24f2882d85d8e09b4f69e. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.620667 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:24.620795 9064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:24.621162 9064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:24.621212 9064 catalog_manager.cc:1259] Loaded cluster ID: 582e90cbf8a14b54ad7a7a6f449e80b7
I20260923 22:08:24.621260 9064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:24.621398 9064 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:24.621472 9064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Loaded TSK: 0
I20260923 22:08:24.621618 9064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:24.655757 8350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:24.657299 8350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:24.658154 8546 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 2 LEARNER]: Refusing update from remote peer dd770ae25cd24f2882d85d8e09b4f69e: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260923 22:08:24.658179 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.658118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46148)
0923 22:08:24.658142 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.658163 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260923 22:08:24.658380 9057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260923 22:08:24.658533 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.658443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.0.0.1:46162)
0923 22:08:24.658487 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.658512 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:08:24.658607 8337 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e -> Peer 58f79a1786984d55a52d7e6a68f85969 (127.1.161.190:38131): Couldn't send request to peer 58f79a1786984d55a52d7e6a68f85969. Status: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 22:08:24.659024 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd770ae25cd24f2882d85d8e09b4f69e. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: NON_VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.659134 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:24.659353 9062 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [term 2 LEADER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.659511 8546 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [term 2 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 95de76c2e6fc401da5ffa317182adc13 (127.1.161.188) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.660121 9065 mvcc.cc:205] Tried to move back new op lower bound from 7332664543877840896 to 7332664543725051904. Current Snapshot: MvccSnapshot[applied={T|T < 7332664543877840896}]
I20260923 22:08:24.660207 9057 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.660221 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.660276 9057 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:24.660290 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:24.660468 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.660543 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:24.660580 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "58f79a1786984d55a52d7e6a68f85969" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38131 } } peers { permanent_uuid: "dd770ae25cd24f2882d85d8e09b4f69e" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45695 } attrs { promote: false } } peers { permanent_uuid: "95de76c2e6fc401da5ffa317182adc13" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 39677 } attrs { promote: false } } }
I20260923 22:08:24.660633 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95de76c2e6fc401da5ffa317182adc13 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:08:24.660684 9067 mvcc.cc:205] Tried to move back new op lower bound from 7332664543877840896 to 7332664543725051904. Current Snapshot: MvccSnapshot[applied={T|T < 7332664543877840896 or (T in {7332664543877840896})}]
I20260923 22:08:24.664528 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.661926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.1.161.129:33689)
0923 22:08:24.662030 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.662032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.662144 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:08:24.662228 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.662230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.662240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.662276 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.662458 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.663003 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.663858 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664201 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664290 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664300 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.664423 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.664461 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260923 22:08:24.664713 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.661887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:33689 (local address 127.0.0.1:55924)
0923 22:08:24.662098 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.662106 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.662163 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.662335 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.662336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.662338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.662383 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.662384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.663078 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.663079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.663776 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.663778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664292 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.664345 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664348 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.664599 (+ 251us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.664644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":115,"thread_start_us":61,"threads_started":1}
I20260923 22:08:24.664878 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.661811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:44831 (local address 127.0.0.1:45816)
0923 22:08:24.661925 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.661932 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.661963 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.662126 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.662127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.662128 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.662162 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.662163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.662844 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.662845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.663785 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.663787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664720 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.664774 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664777 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.664826 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.664848 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":1}
I20260923 22:08:24.664947 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.661885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46331 (local address 127.0.0.1:58132)
0923 22:08:24.662053 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.662059 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.662095 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.662357 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.662358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.662359 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.662404 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.662405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.663229 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.663230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.664131 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.664133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664744 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.664806 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664809 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.664852 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.664885 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260923 22:08:24.664942 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.661874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.1.161.130:44831)
0923 22:08:24.661984 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.661986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.661994 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:24.662030 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.662031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.662041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.662122 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.662207 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.662768 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.663873 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664506 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664581 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664590 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.664844 (+ 254us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.664883 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":66,"threads_started":1}
I20260923 22:08:24.665097 8787 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8088eea00d415aa66726bec390382f (DEFAULT_TABLE table=test-table [id=8c4bdc4eaa07455a8cc40eaecf199399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:24.665158 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.662159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.1.161.131:46331)
0923 22:08:24.662253 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.662255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.662261 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:24.662281 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.662282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.662290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.662500 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.662535 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.663130 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664206 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.664667 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.664745 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.664753 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.665071 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.665103 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":49,"threads_started":1}
I20260923 22:08:24.665194 8787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8088eea00d415aa66726bec390382f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:24.665545 8651 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8088eea00d415aa66726bec390382f (DEFAULT_TABLE table=test-table [id=8c4bdc4eaa07455a8cc40eaecf199399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:24.665669 8651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8088eea00d415aa66726bec390382f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:24.666483 9077 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Bootstrap starting.
I20260923 22:08:24.666596 8923 tablet_service.cc:1467] Processing CreateTablet for tablet 5b8088eea00d415aa66726bec390382f (DEFAULT_TABLE table=test-table [id=8c4bdc4eaa07455a8cc40eaecf199399]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:24.666690 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b8088eea00d415aa66726bec390382f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:24.666986 9077 tablet_bootstrap.cc:654] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:24.667250 9078 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: Bootstrap starting.
I20260923 22:08:24.667613 9077 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: No bootstrap required, opened a new log
I20260923 22:08:24.667640 9078 tablet_bootstrap.cc:654] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:24.667665 9077 ts_tablet_manager.cc:1397] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 22:08:24.667778 9077 raft_consensus.cc:348] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.667840 9077 raft_consensus.cc:374] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:24.667857 9077 raft_consensus.cc:729] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fe4644f0f84222a0ac1d6b641cb2ab, State: Initialized, Role: FOLLOWER
I20260923 22:08:24.667898 9077 consensus_queue.cc:260] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.667989 9081 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Bootstrap starting.
I20260923 22:08:24.668017 9077 ts_tablet_manager.cc:1428] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:24.668305 9078 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: No bootstrap required, opened a new log
I20260923 22:08:24.668329 9081 tablet_bootstrap.cc:654] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:24.668350 9078 ts_tablet_manager.cc:1397] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:24.668448 9078 raft_consensus.cc:348] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.668504 9078 raft_consensus.cc:374] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:24.668516 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.668206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:51625)
0923 22:08:24.668458 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.668488 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":65,"threads_started":1}
I20260923 22:08:24.668522 9078 raft_consensus.cc:729] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c49f2022bdf46ea82a080009276e4c5, State: Initialized, Role: FOLLOWER
I20260923 22:08:24.668565 9078 consensus_queue.cc:260] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.668701 9078 ts_tablet_manager.cc:1428] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:24.668973 9081 tablet_bootstrap.cc:492] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: No bootstrap required, opened a new log
I20260923 22:08:24.669019 9081 ts_tablet_manager.cc:1397] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:24.669108 9081 raft_consensus.cc:348] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.669162 9081 raft_consensus.cc:374] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:24.669178 9081 raft_consensus.cc:729] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f6337539c245fbae3be625407f769a, State: Initialized, Role: FOLLOWER
I20260923 22:08:24.669186 8719 heartbeater.cc:503] Master 127.1.161.189:45695 was elected leader, sending a full tablet report...
I20260923 22:08:24.669226 9081 consensus_queue.cc:260] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.669268 8855 heartbeater.cc:503] Master 127.1.161.189:45695 was elected leader, sending a full tablet report...
I20260923 22:08:24.669394 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.669314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:60179)
0923 22:08:24.669346 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.669373 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:24.669507 9081 ts_tablet_manager.cc:1428] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:24.670107 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.668962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:53995)
0923 22:08:24.670018 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.670064 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1017,"thread_start_us":63,"threads_started":1}
I20260923 22:08:24.669852 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.669664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:43055)
0923 22:08:24.669770 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.669796 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
I20260923 22:08:24.670315 8991 heartbeater.cc:503] Master 127.1.161.189:45695 was elected leader, sending a full tablet report...
I20260923 22:08:24.670459 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.670382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:57657)
0923 22:08:24.670414 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.670436 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:08:24.734242 8992 tablet.cc:2369] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:08:24.737710 8720 tablet.cc:2369] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:24.813048 9085 raft_consensus.cc:482] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:24.813127 9085 raft_consensus.cc:504] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.813482 9085 leader_election.cc:288] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831), b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:24.816439 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.813760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46331 (local address 127.1.161.129:37439)
0923 22:08:24.813931 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.813941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.814020 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.814175 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.814175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.814177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.814216 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.814217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.814805 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.814806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.815659 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.815661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.816206 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.816208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.816267 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.816270 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.816319 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.816347 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":17,"thread_start_us":125,"threads_started":1}
I20260923 22:08:24.816471 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.813833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:37439 (local address 127.1.161.131:46331)
0923 22:08:24.813901 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.813903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.814030 (+ 127us) server_negotiation.cc:407] Connection header received
0923 22:08:24.814080 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.814081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.814090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.814147 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.814267 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.814661 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.815750 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.816116 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.816180 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.816188 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.816350 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.816391 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:24.816579 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.813727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:41349 (local address 127.1.161.130:44831)
0923 22:08:24.813778 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.813780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.813789 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:24.813970 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.813971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.813981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.814041 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:24.814188 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.814644 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.815788 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.816136 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.816232 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.816242 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.816445 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.816487 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:24.816665 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8088eea00d415aa66726bec390382f" candidate_uuid: "6c49f2022bdf46ea82a080009276e4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f6337539c245fbae3be625407f769a" is_pre_election: true
I20260923 22:08:24.816763 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.813637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:44831 (local address 127.1.161.129:41349)
0923 22:08:24.813714 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.813727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:24.813876 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:24.814060 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:24.814061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:24.814064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:24.814123 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.814124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.814754 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.814755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.815664 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:24.815667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.816261 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.816262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:24.816354 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.816358 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:24.816421 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:24.816457 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":28}
I20260923 22:08:24.816761 8943 raft_consensus.cc:2454] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c49f2022bdf46ea82a080009276e4c5 in term 0.
I20260923 22:08:24.816969 8605 leader_election.cc:302] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c49f2022bdf46ea82a080009276e4c5, b7f6337539c245fbae3be625407f769a; no voters:
I20260923 22:08:24.817081 9085 raft_consensus.cc:2781] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:24.817111 9085 raft_consensus.cc:482] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:24.817047 8807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8088eea00d415aa66726bec390382f" candidate_uuid: "6c49f2022bdf46ea82a080009276e4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" is_pre_election: true
I20260923 22:08:24.817130 9085 raft_consensus.cc:3037] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:24.817158 8807 raft_consensus.cc:2454] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c49f2022bdf46ea82a080009276e4c5 in term 0.
I20260923 22:08:24.817790 9085 raft_consensus.cc:504] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.817920 9085 leader_election.cc:288] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [CANDIDATE]: Term 1 election: Requested vote from peers 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831), b7f6337539c245fbae3be625407f769a (127.1.161.131:46331)
I20260923 22:08:24.818116 8807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8088eea00d415aa66726bec390382f" candidate_uuid: "6c49f2022bdf46ea82a080009276e4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab"
I20260923 22:08:24.818199 8807 raft_consensus.cc:3037] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:24.818174 8943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b8088eea00d415aa66726bec390382f" candidate_uuid: "6c49f2022bdf46ea82a080009276e4c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f6337539c245fbae3be625407f769a"
I20260923 22:08:24.818266 8943 raft_consensus.cc:3037] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:24.818778 8943 raft_consensus.cc:2454] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c49f2022bdf46ea82a080009276e4c5 in term 1.
I20260923 22:08:24.818799 8807 raft_consensus.cc:2454] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c49f2022bdf46ea82a080009276e4c5 in term 1.
I20260923 22:08:24.818977 8605 leader_election.cc:302] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fe4644f0f84222a0ac1d6b641cb2ab, 6c49f2022bdf46ea82a080009276e4c5; no voters:
I20260923 22:08:24.819069 9085 raft_consensus.cc:2781] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:24.819211 9085 raft_consensus.cc:686] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 6c49f2022bdf46ea82a080009276e4c5, State: Running, Role: LEADER
I20260923 22:08:24.819291 9085 consensus_queue.cc:237] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } }
I20260923 22:08:24.819479 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.819383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:40881)
0923 22:08:24.819437 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.819460 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:08:24.819958 8350 catalog_manager.cc:5495] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "6c49f2022bdf46ea82a080009276e4c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c49f2022bdf46ea82a080009276e4c5" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 33689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:24.844864 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.842836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.1.161.129:33689)
0923 22:08:24.842886 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:24.842887 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:24.842895 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:24.842953 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:24.842954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:24.842970 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:24.843061 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:24.843196 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.843563 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.844207 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:24.844368 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:24.844436 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:24.844595 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:24.844681 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:24.844757 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:24.844792 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:08:24.844808 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:08:24.845789 8943 raft_consensus.cc:1261] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Refusing update from remote peer 6c49f2022bdf46ea82a080009276e4c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:24.845921 8807 raft_consensus.cc:1261] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Refusing update from remote peer 6c49f2022bdf46ea82a080009276e4c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:24.846033 9085 consensus_queue.cc:1035] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f6337539c245fbae3be625407f769a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:24.846321 9085 consensus_queue.cc:1035] T 5b8088eea00d415aa66726bec390382f P 6c49f2022bdf46ea82a080009276e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fe4644f0f84222a0ac1d6b641cb2ab" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 44831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:24.846743 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.846602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:60781)
0923 22:08:24.846678 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.846712 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 22:08:24.847785 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.847585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:44093)
0923 22:08:24.847725 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.847755 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260923 22:08:24.848085 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.847970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.131:47573)
0923 22:08:24.848037 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.848056 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260923 22:08:24.848551 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.848458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:44865)
0923 22:08:24.848495 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.848524 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 22:08:24.848888 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.848789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.129:48525)
0923 22:08:24.848822 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.848860 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:08:24.849185 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.849115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:38131 (local address 127.1.161.130:54275)
0923 22:08:24.849143 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.849167 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:08:24.850303 8350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:24.850387 8350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:24.851516 8350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd770ae25cd24f2882d85d8e09b4f69e: Sending DeleteTablet for 3 replicas of tablet 5b8088eea00d415aa66726bec390382f
I20260923 22:08:24.851850 8651 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b8088eea00d415aa66726bec390382f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:24 UTC) from {username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:55924
I20260923 22:08:24.851888 8787 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b8088eea00d415aa66726bec390382f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:24 UTC) from {username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:45816
I20260923 22:08:24.852046 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8588
I20260923 22:08:24.852066 9103 tablet_replica.cc:330] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: stopping tablet replica
I20260923 22:08:24.852061 8923 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b8088eea00d415aa66726bec390382f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:24 UTC) from {username='oryx', principal='oryx/127.1.161.189@KRBTEST.COM'} at 127.0.0.1:58132
I20260923 22:08:24.852109 9103 raft_consensus.cc:2229] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:24.852160 9103 raft_consensus.cc:2258] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:24.852631 9103 ts_tablet_manager.cc:1905] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:24.854357 9103 ts_tablet_manager.cc:1918] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:24.854404 9103 log.cc:1196] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/5b8088eea00d415aa66726bec390382f
I20260923 22:08:24.854816 9103 ts_tablet_manager.cc:1939] T 5b8088eea00d415aa66726bec390382f P 56fe4644f0f84222a0ac1d6b641cb2ab: Deleting consensus metadata
I20260923 22:08:24.854836 9104 tablet_replica.cc:330] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: stopping tablet replica
I20260923 22:08:24.854919 9104 raft_consensus.cc:2229] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:24.854966 9104 raft_consensus.cc:2258] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:24.855247 9104 ts_tablet_manager.cc:1905] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:24.855234 8335 catalog_manager.cc:4841] TS 56fe4644f0f84222a0ac1d6b641cb2ab (127.1.161.130:44831): tablet 5b8088eea00d415aa66726bec390382f (table test-table [id=8c4bdc4eaa07455a8cc40eaecf199399]) successfully deleted
I20260923 22:08:24.856670 9104 ts_tablet_manager.cc:1918] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:24.856727 9104 log.cc:1196] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/5b8088eea00d415aa66726bec390382f
I20260923 22:08:24.856949 9104 ts_tablet_manager.cc:1939] T 5b8088eea00d415aa66726bec390382f P b7f6337539c245fbae3be625407f769a: Deleting consensus metadata
I20260923 22:08:24.857234 8335 catalog_manager.cc:4841] TS b7f6337539c245fbae3be625407f769a (127.1.161.131:46331): tablet 5b8088eea00d415aa66726bec390382f (table test-table [id=8c4bdc4eaa07455a8cc40eaecf199399]) successfully deleted
I20260923 22:08:24.860159 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8724
I20260923 22:08:24.860268 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:24.860150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:33689 (local address 127.0.0.1:55946)
0923 22:08:24.860211 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:24.860237 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 22:08:24.860654 8334 catalog_manager.cc:4584] TS 6c49f2022bdf46ea82a080009276e4c5 (127.1.161.129:33689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b8088eea00d415aa66726bec390382f: Network error: Client connection negotiation failed: client connection to 127.1.161.129:33689: connect: Connection refused (error 111)
I20260923 22:08:24.865595 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8860
I20260923 22:08:24.870278 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8216
I20260923 22:08:24.874377 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 8404
2026-09-23T22:08:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4352 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:24 dist-test-slave-npvd krb5kdc[9107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:24 dist-test-slave-npvd krb5kdc[9107](info): set up 2 sockets
Sep 23 22:08:24 dist-test-slave-npvd krb5kdc[9107](info): commencing operation
krb5kdc: starting...
W20260923 22:08:25.791005 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.896s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:25 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab.
Entry for principal oryx/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab.
I20260923 22:08:25.894277 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46873
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab
--principal=oryx/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:46873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:25.967595 9120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:25.967747 9120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:25.967765 9120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:25.968724 9120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:25.968760 9120 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 22:08:25.968771 9120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:25.968782 9120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:25.968793 9120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:25.968803 9120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:25.969967 9120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:46873
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.190.keytab
--principal=oryx/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:25.970108 9120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:25.970244 9120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:25.974589 9128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:25.974620 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:25.974589 9125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:25.974845 9120 server_base.cc:1034] running on GCE node
I20260923 22:08:25.975076 9120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:25.975363 9120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:25.976496 9120 hybrid_clock.cc:648] HybridClock initialized: now 1790201305976484 us; error 25 us; skew 500 ppm
Sep 23 22:08:25 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201305, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:25.978637 9120 init.cc:377] Logged in from keytab as oryx/127.1.161.190@KRBTEST.COM (short username oryx)
I20260923 22:08:25.979004 9120 webserver.cc:460] Webserver started at http://127.1.161.190:40577/ using document root <none> and password file <none>
I20260923 22:08:25.979247 9120 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:25.979290 9120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:25.979411 9120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:25.980228 9120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "b02f7819601d46ebb6fc0dc3ac40565c"
format_stamp: "Formatted at 2026-09-23 22:08:25 on dist-test-slave-npvd"
server_key: "0ab11059f3a6a690e585f43187965b58"
server_key_iv: "62133c7a47f3517ca3101f6c1446eada"
server_key_version: "encryptionkey@0"
I20260923 22:08:25.980530 9120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "b02f7819601d46ebb6fc0dc3ac40565c"
format_stamp: "Formatted at 2026-09-23 22:08:25 on dist-test-slave-npvd"
server_key: "0ab11059f3a6a690e585f43187965b58"
server_key_iv: "62133c7a47f3517ca3101f6c1446eada"
server_key_version: "encryptionkey@0"
I20260923 22:08:25.981624 9120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:25.982165 9135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:25.982308 9120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:25.982393 9120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "b02f7819601d46ebb6fc0dc3ac40565c"
format_stamp: "Formatted at 2026-09-23 22:08:25 on dist-test-slave-npvd"
server_key: "0ab11059f3a6a690e585f43187965b58"
server_key_iv: "62133c7a47f3517ca3101f6c1446eada"
server_key_version: "encryptionkey@0"
I20260923 22:08:25.982462 9120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:25.989063 9120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:25.990504 9120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:25.990653 9120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:25.994412 9120 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:46873
I20260923 22:08:25.994561 9197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:46873 every 8 connection(s)
I20260923 22:08:25.994755 9120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:25.995407 9198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:25.997325 9198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Bootstrap starting.
I20260923 22:08:25.997807 9198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:25.998067 9198 log.cc:824] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:25.998325 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9120
I20260923 22:08:25.998420 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:25.998579 1670 external_mini_cluster.cc:1428] Setting key 209b3a73d98c8cbacfafde1badbc7172
I20260923 22:08:25.998750 9198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: No bootstrap required, opened a new log
I20260923 22:08:26.001173 9198 raft_consensus.cc:348] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } }
I20260923 22:08:26.001303 9198 raft_consensus.cc:374] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.001331 9198 raft_consensus.cc:729] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02f7819601d46ebb6fc0dc3ac40565c, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.001420 9198 consensus_queue.cc:260] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } }
I20260923 22:08:26.001473 9198 raft_consensus.cc:388] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:26.001495 9198 raft_consensus.cc:482] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:26.001530 9198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.002050 9198 raft_consensus.cc:504] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } }
I20260923 22:08:26.002130 9198 leader_election.cc:302] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b02f7819601d46ebb6fc0dc3ac40565c; no voters:
I20260923 22:08:26.002280 9198 leader_election.cc:288] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:26.002305 9203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:26.002385 9203 raft_consensus.cc:686] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [term 1 LEADER]: Becoming Leader. State: Replica: b02f7819601d46ebb6fc0dc3ac40565c, State: Running, Role: LEADER
I20260923 22:08:26.002457 9203 consensus_queue.cc:237] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } }
I20260923 22:08:26.002481 9198 sys_catalog.cc:564] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:26.002933 9204 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } } }
I20260923 22:08:26.002928 9205 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b02f7819601d46ebb6fc0dc3ac40565c. Latest consensus state: current_term: 1 leader_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02f7819601d46ebb6fc0dc3ac40565c" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46873 } } }
I20260923 22:08:26.003012 9205 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:26.003136 9204 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:26.003255 9209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:26.003669 9209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:26.005293 9209 catalog_manager.cc:1347] Generated new cluster ID: 7de1656f1a6e4c3c9a52042de74bc091
I20260923 22:08:26.005330 9209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.009168 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:25.999145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.1.161.190:46873)
0923 22:08:25.999355 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:25.999362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:25.999393 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:25.999976 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:25.999983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.000034 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.000156 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.000651 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.001735 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.002600 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.003089 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.003210 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.005903 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.005906 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.005906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.005915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.007765 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.008175 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.008177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.008178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.008258 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.008455 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.008456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.008456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.008679 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.008802 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.008907 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.009008 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260923 22:08:26.016908 9209 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:26.017371 9209 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:26.021610 9209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Generated new TSK 0
I20260923 22:08:26.021763 9209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
I20260923 22:08:26.047940 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:26.117671 9224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:26.117853 9224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:26.117885 9224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:26.118856 9224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:26.118922 9224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:26.118953 9224 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:26.118983 9224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:26.120245 9224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:26.120472 9224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:26.120664 9224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:26.124884 9224 server_base.cc:1034] running on GCE node
W20260923 22:08:26.124881 9232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:26.124861 9230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:26.124851 9229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:26.125200 9224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:26.125407 9224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:26.126566 9224 hybrid_clock.cc:648] HybridClock initialized: now 1790201306126542 us; error 36 us; skew 500 ppm
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:26.128786 9224 init.cc:377] Logged in from keytab as oryx/127.1.161.129@KRBTEST.COM (short username oryx)
I20260923 22:08:26.129186 9224 webserver.cc:460] Webserver started at http://127.1.161.129:40995/ using document root <none> and password file <none>
I20260923 22:08:26.129334 9224 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:26.129366 9224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:26.129436 9224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:26.130227 9224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "d2372e9f613e489c81b94b43eeaa1d14"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "44d1b8b78d4248cf3542efe8dd5973f0"
server_key_iv: "43ffd466f672dda57e4a8ccb8aec7ef7"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.130463 9224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "d2372e9f613e489c81b94b43eeaa1d14"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "44d1b8b78d4248cf3542efe8dd5973f0"
server_key_iv: "43ffd466f672dda57e4a8ccb8aec7ef7"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.131520 9224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:26.132064 9239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.132216 9224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.132288 9224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d2372e9f613e489c81b94b43eeaa1d14"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "44d1b8b78d4248cf3542efe8dd5973f0"
server_key_iv: "43ffd466f672dda57e4a8ccb8aec7ef7"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.132332 9224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:26.145645 9224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:26.147703 9224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:26.147791 9224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:26.148125 9224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:26.148165 9224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.148183 9224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:26.148195 9224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.154241 9224 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:38425
I20260923 22:08:26.154286 9354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:38425 every 8 connection(s)
I20260923 22:08:26.154527 9224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:26.157294 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.148548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:34533 (local address 127.1.161.190:46873)
0923 22:08:26.148624 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.148627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.148791 (+ 164us) server_negotiation.cc:407] Connection header received
0923 22:08:26.149119 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.149120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.149133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.149207 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.149918 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.150627 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.151951 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.152228 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.152288 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.154539 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.154540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.154540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.154546 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.155937 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.156343 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.156345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.156345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.156399 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.156668 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.156669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.156669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.156783 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.156858 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.157136 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.157214 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:08:26.157402 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.148483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.129:34533)
0923 22:08:26.148701 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.148723 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.149040 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.149243 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.149272 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.149585 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.149820 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.149826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.150737 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.150738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.151876 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.151879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.152307 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.152309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.152456 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.152867 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.152877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.154448 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.156037 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.156043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.156045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.156258 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.156487 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.156488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.156488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.156594 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.156863 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.156869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.156995 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.157108 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.157230 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":68,"threads_started":1}
I20260923 22:08:26.157815 9355 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46873
I20260923 22:08:26.157904 9355 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:26.158041 9355 heartbeater.cc:511] Master 127.1.161.190:46873 requested a full tablet report, sending...
I20260923 22:08:26.158550 9151 ts_manager.cc:194] Registered new tserver with Master: d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425)
I20260923 22:08:26.159241 9151 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:34533
I20260923 22:08:26.161662 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9224
I20260923 22:08:26.161785 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:26.161972 1670 external_mini_cluster.cc:1428] Setting key 6efb929da76862e51f68c5c2f77359da
I20260923 22:08:26.163883 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.159238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:42723 (local address 127.1.161.190:46873)
0923 22:08:26.159307 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.159309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.159317 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:26.159334 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.159335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.159343 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.159391 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.159713 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.160091 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.160715 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.160917 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.160996 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.161697 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.161698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.161698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.161704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.162902 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.163222 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.163223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.163223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.163267 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.163450 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.163451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.163451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.163542 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.163607 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.163751 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.163809 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:08:26.163887 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.158675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.129:42723)
0923 22:08:26.158778 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.158787 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.158831 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.159396 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.159397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.159569 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.159623 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.159625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.160195 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.160196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.160637 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.160639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.160996 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.160996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.161052 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.161368 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.161372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.161611 (+ 239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.162981 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.162981 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.162982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.163129 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.163329 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.163329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.163330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.163377 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.163612 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.163613 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.163655 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.163730 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.163810 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
WARNING: no policy specified for oryx/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
I20260923 22:08:26.185720 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:26.256299 9364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:26.256464 9364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:26.256486 9364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:26.257550 9364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:26.257584 9364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:26.257601 9364 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:26.257615 9364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:26.258875 9364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:26.259052 9364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:26.259224 9364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:26.263407 9364 server_base.cc:1034] running on GCE node
W20260923 22:08:26.263389 9369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:26.263394 9370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:26.263396 9372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:26.263748 9364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:26.263999 9364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:26.265159 9364 hybrid_clock.cc:648] HybridClock initialized: now 1790201306265146 us; error 23 us; skew 500 ppm
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:26.267453 9364 init.cc:377] Logged in from keytab as oryx/127.1.161.130@KRBTEST.COM (short username oryx)
I20260923 22:08:26.267851 9364 webserver.cc:460] Webserver started at http://127.1.161.130:33671/ using document root <none> and password file <none>
I20260923 22:08:26.268052 9364 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:26.268113 9364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:26.268216 9364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:26.269040 9364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "e30528f598da45aeb45d84e48ab00055"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "b6a6b205c06dcb480a47a9a2294040a6"
server_key_iv: "1cdbb67f30ec766ae887baf1e22c0b2c"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.269366 9364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "e30528f598da45aeb45d84e48ab00055"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "b6a6b205c06dcb480a47a9a2294040a6"
server_key_iv: "1cdbb67f30ec766ae887baf1e22c0b2c"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.270423 9364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:26.271044 9379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.271246 9364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.271327 9364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "e30528f598da45aeb45d84e48ab00055"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "b6a6b205c06dcb480a47a9a2294040a6"
server_key_iv: "1cdbb67f30ec766ae887baf1e22c0b2c"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.271384 9364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:26.281468 9364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:26.282930 9364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:26.283068 9364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:26.283550 9364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:26.283612 9364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.283636 9364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:26.283651 9364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.289832 9364 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:40143
I20260923 22:08:26.289882 9494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:40143 every 8 connection(s)
I20260923 22:08:26.290107 9364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:26.292872 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.283843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:56595 (local address 127.1.161.190:46873)
0923 22:08:26.283879 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.283881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.284043 (+ 162us) server_negotiation.cc:407] Connection header received
0923 22:08:26.284432 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.284434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.284445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.284493 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.285088 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.285809 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.287438 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.287681 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.287761 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.289820 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.289821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.289822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.289829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.291215 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.291799 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.291801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.291801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.291865 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.292165 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.292166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.292166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.292290 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.292393 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.292704 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.292782 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:26.293066 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.283812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.130:56595)
0923 22:08:26.283924 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.283951 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.284337 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.284542 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.284546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.284790 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.284996 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.285000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.285897 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.285898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.287354 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.287357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.287782 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.287783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.287917 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.288349 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.288358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.289738 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.291300 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.291305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.291307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.291685 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.291953 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.291954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.291954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.292069 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.292384 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.292390 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.292518 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.292683 (+ 165us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.292827 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
I20260923 22:08:26.293519 9495 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46873
I20260923 22:08:26.293599 9495 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:26.293778 9495 heartbeater.cc:511] Master 127.1.161.190:46873 requested a full tablet report, sending...
I20260923 22:08:26.294127 9151 ts_manager.cc:194] Registered new tserver with Master: e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:26.294581 9151 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:56595
I20260923 22:08:26.299561 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9364
I20260923 22:08:26.299691 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:26.299960 1670 external_mini_cluster.cc:1428] Setting key 9c8c982fea47e162206d8388036a6a8c
I20260923 22:08:26.300520 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.294464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:45575 (local address 127.1.161.190:46873)
0923 22:08:26.294498 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.294500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.294557 (+ 57us) server_negotiation.cc:407] Connection header received
0923 22:08:26.294619 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.294620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.294626 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.294707 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.295094 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.295579 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.296666 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.296870 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.296963 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.297900 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.297902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.297902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.297908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.299054 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.299624 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.299626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.299626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.299683 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.299897 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.299898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.299898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.300111 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.300195 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.300352 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.300410 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:08:26.300531 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.294390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.130:45575)
0923 22:08:26.294506 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.294515 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.294565 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.294705 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.294705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.294951 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.295002 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.295003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.295743 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.295744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.296527 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.296530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.296969 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.296970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.297033 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.297397 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.297401 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.297775 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.299305 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.299306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.299307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.299525 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.299741 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.299742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.299743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.299804 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.300205 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.300206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.300251 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.300332 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.300421 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
I20260923 22:08:26.326296 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:46873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:26.397665 9504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:26.397850 9504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:26.397882 9504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:26.398998 9504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:26.399075 9504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:26.399108 9504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 22:08:26.399153 9504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:26.400470 9504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.161.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:26.400696 9504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:26.400882 9504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:26.405346 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:26.405356 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:26.405359 9504 server_base.cc:1034] running on GCE node
W20260923 22:08:26.405336 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:26.405709 9504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:26.405910 9504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:26.407081 9504 hybrid_clock.cc:648] HybridClock initialized: now 1790201306407078 us; error 37 us; skew 500 ppm
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:26.409142 9504 init.cc:377] Logged in from keytab as oryx/127.1.161.131@KRBTEST.COM (short username oryx)
I20260923 22:08:26.409484 9504 webserver.cc:460] Webserver started at http://127.1.161.131:38543/ using document root <none> and password file <none>
I20260923 22:08:26.409696 9504 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:26.409763 9504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:26.409894 9504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:26.410681 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "6c231298be9246f699b7efc300b21f02"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "38d7b0f5da26f2f5e0247cc21da8d4b1"
server_key_iv: "fc63dd3f2f1f697b30c449207844079b"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.410989 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "6c231298be9246f699b7efc300b21f02"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "38d7b0f5da26f2f5e0247cc21da8d4b1"
server_key_iv: "fc63dd3f2f1f697b30c449207844079b"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.412139 9504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:26.412725 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.412889 9504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:26.412972 9504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "6c231298be9246f699b7efc300b21f02"
format_stamp: "Formatted at 2026-09-23 22:08:26 on dist-test-slave-npvd"
server_key: "38d7b0f5da26f2f5e0247cc21da8d4b1"
server_key_iv: "fc63dd3f2f1f697b30c449207844079b"
server_key_version: "encryptionkey@0"
I20260923 22:08:26.413058 9504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:26.433019 9504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:26.434568 9504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:26.434702 9504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:26.435132 9504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:26.435231 9504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.435297 9504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:26.435318 9504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.442059 9504 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:33011
I20260923 22:08:26.442204 9634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:33011 every 8 connection(s)
I20260923 22:08:26.442438 9504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:26.444334 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.435613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:48583 (local address 127.1.161.190:46873)
0923 22:08:26.435691 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.435693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.435881 (+ 188us) server_negotiation.cc:407] Connection header received
0923 22:08:26.436491 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.436492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.436524 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.436597 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.437441 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.438177 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.439247 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.439438 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.439525 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.441696 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.441697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.441697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.441703 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.443223 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.443598 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.443599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.443599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.443653 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.443878 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.443879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.443880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.443978 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.444036 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.444222 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.444275 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 22:08:26.444458 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.435590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.131:48583)
0923 22:08:26.435747 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.435792 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.436394 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.436668 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.436677 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.437063 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.437345 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.437350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.438261 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.438262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.439012 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.439014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.439546 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.439547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.439684 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.440148 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.440157 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.441594 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.443287 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.443290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.443292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.443514 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.443723 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.443724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.443724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.443808 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.444029 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.444031 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.444119 (+ 88us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.444186 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.444294 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260923 22:08:26.444953 9635 heartbeater.cc:347] Connected to a master server at 127.1.161.190:46873
I20260923 22:08:26.445111 9635 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:26.445412 9635 heartbeater.cc:511] Master 127.1.161.190:46873 requested a full tablet report, sending...
I20260923 22:08:26.445827 9151 ts_manager.cc:194] Registered new tserver with Master: 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011)
I20260923 22:08:26.446318 9151 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:48583
I20260923 22:08:26.450240 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9504
I20260923 22:08:26.450377 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:26.450555 1670 external_mini_cluster.cc:1428] Setting key 12fd9adff00cd8dfca0e56e83782fe9b
I20260923 22:08:26.451303 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:26.451557 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.445960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:44495 (local address 127.1.161.190:46873)
0923 22:08:26.446000 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.446001 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.446078 (+ 77us) server_negotiation.cc:407] Connection header received
0923 22:08:26.446125 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.446126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.446134 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.446192 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.446487 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.446942 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.447804 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.448014 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.448107 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.448998 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.448999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.448999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.449005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.450325 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.450730 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.450731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.450731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.450784 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.451027 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.451028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.451028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.451134 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.451253 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.451409 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.451476 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:26.451589 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.445883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.1.161.131:44495)
0923 22:08:26.446000 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.446026 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.446088 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.446195 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.446195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.446340 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.446383 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.446384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.447117 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.447118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.447682 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.447684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.448110 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.448111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.448188 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.448626 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.448630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.448884 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.450447 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.450448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.450448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.450625 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.450882 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.450883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.450884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.450944 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.451248 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.451249 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.451290 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.451385 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.451477 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:26.471125 9202 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:26.466263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.1.161.190:46873)
0923 22:08:26.466325 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.466327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.466347 (+ 20us) server_negotiation.cc:407] Connection header received
0923 22:08:26.466367 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.466368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.466383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.466441 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.466991 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.467519 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.468164 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.468328 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.468392 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.471069 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34358: BlockingRecv error: recv got EOF from 127.0.0.1:34358 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201306, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.478475 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.471843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.1.161.190:46873)
0923 22:08:26.471902 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.471904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.471943 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:08:26.472006 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.472006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.472013 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.472055 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.472643 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.473069 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.473808 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.473985 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.474045 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.476283 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.476284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.476284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.476290 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.477637 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.477966 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.477966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.477966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.478006 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.478159 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.478160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.478160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.478244 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.478303 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.478385 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.478430 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:26.479766 9151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:26.481230 9151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:26.486915 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:38425 (local address 127.0.0.1:37886)
0923 22:08:26.483208 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.483215 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.483256 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.483781 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.483784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.483786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.483855 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.483857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.484714 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.484715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.485765 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.485767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.486679 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.486680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.486746 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.486767 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.486808 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.486850 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260923 22:08:26.487020 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:40143 (local address 127.0.0.1:50038)
0923 22:08:26.483316 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.483325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.483371 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.483787 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.483788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.483790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.483857 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.483859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.485015 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.485016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.485880 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.485881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.486605 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.486606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.486748 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.486775 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.486849 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.486896 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":3,"thread_start_us":40,"threads_started":1}
I20260923 22:08:26.487134 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.1.161.130:40143)
0923 22:08:26.483595 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.483599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.483609 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:26.483647 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.483660 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.483676 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.483926 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.483982 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.484934 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.485979 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.486493 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.486599 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.486745 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.486863 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.486903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260923 22:08:26.487488 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.1.161.129:38425)
0923 22:08:26.483629 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.483633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.483641 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:26.483668 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.483670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.483680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.483778 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.483932 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.484641 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.485847 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.486583 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.487027 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.487114 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.487372 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.487408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260923 22:08:26.487915 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 11935054b0c64abcbd29860314849b21 (DEFAULT_TABLE table=test-table [id=b3868e748ce94d998ebb9a73adf0211a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.488060 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33011 (local address 127.0.0.1:35712)
0923 22:08:26.483134 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.483184 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.483243 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.483856 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.483857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.483859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.483889 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.483891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.485951 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.485951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.486999 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.487002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.487837 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.487838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.487895 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.487900 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.487971 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.488007 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":51,"threads_started":1}
I20260923 22:08:26.488088 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.483244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.1.161.131:33011)
0923 22:08:26.483675 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.483678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.483686 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:26.483715 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.483716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.483734 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.483798 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.483953 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.484933 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.487082 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.487759 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.487820 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.487874 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.487984 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.488017 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":60,"threads_started":1}
I20260923 22:08:26.487995 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 11935054b0c64abcbd29860314849b21 (DEFAULT_TABLE table=test-table [id=b3868e748ce94d998ebb9a73adf0211a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.488291 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11935054b0c64abcbd29860314849b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.488319 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11935054b0c64abcbd29860314849b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.488868 9569 tablet_service.cc:1467] Processing CreateTablet for tablet 11935054b0c64abcbd29860314849b21 (DEFAULT_TABLE table=test-table [id=b3868e748ce94d998ebb9a73adf0211a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.489168 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11935054b0c64abcbd29860314849b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.491225 9660 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Bootstrap starting.
I20260923 22:08:26.491740 9660 tablet_bootstrap.cc:654] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.491941 9660 log.cc:824] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:26.492542 9660 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: No bootstrap required, opened a new log
I20260923 22:08:26.492575 9662 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Bootstrap starting.
I20260923 22:08:26.492596 9660 ts_tablet_manager.cc:1397] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:26.493139 9662 tablet_bootstrap.cc:654] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.493494 9662 log.cc:824] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:26.494146 9662 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: No bootstrap required, opened a new log
I20260923 22:08:26.494207 9662 ts_tablet_manager.cc:1397] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:26.494696 9660 raft_consensus.cc:348] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.494843 9660 raft_consensus.cc:374] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.494872 9660 raft_consensus.cc:729] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.494997 9660 consensus_queue.cc:260] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.495261 9660 ts_tablet_manager.cc:1428] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:26.495332 9355 heartbeater.cc:503] Master 127.1.161.190:46873 was elected leader, sending a full tablet report...
I20260923 22:08:26.495443 9663 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Bootstrap starting.
I20260923 22:08:26.495970 9663 tablet_bootstrap.cc:654] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.496151 9662 raft_consensus.cc:348] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.496313 9662 raft_consensus.cc:374] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.496309 9663 log.cc:824] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:26.496353 9662 raft_consensus.cc:729] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30528f598da45aeb45d84e48ab00055, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.496466 9662 consensus_queue.cc:260] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.496752 9662 ts_tablet_manager.cc:1428] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:08:26.496757 9495 heartbeater.cc:503] Master 127.1.161.190:46873 was elected leader, sending a full tablet report...
I20260923 22:08:26.497025 9663 tablet_bootstrap.cc:492] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: No bootstrap required, opened a new log
I20260923 22:08:26.497112 9663 ts_tablet_manager.cc:1397] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:26.498546 9663 raft_consensus.cc:348] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.498697 9663 raft_consensus.cc:374] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.498721 9663 raft_consensus.cc:729] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.498813 9663 consensus_queue.cc:260] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.499015 9663 ts_tablet_manager.cc:1428] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:26.499064 9635 heartbeater.cc:503] Master 127.1.161.190:46873 was elected leader, sending a full tablet report...
W20260923 22:08:26.540685 9496 tablet.cc:2369] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:26.651443 9668 raft_consensus.cc:482] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:26.651551 9668 raft_consensus.cc:504] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.651844 9668 leader_election.cc:288] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:26.654359 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.651986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:38425 (local address 127.1.161.131:32953)
0923 22:08:26.652022 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.652032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.652068 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.652231 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.652232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.652234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.652304 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.652306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.652899 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.652900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.653559 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.653561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.654079 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.654080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.654184 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.654187 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.654232 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.654261 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":1}
I20260923 22:08:26.654377 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.652010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:32953 (local address 127.1.161.129:38425)
0923 22:08:26.652053 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.652055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.652064 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:26.652135 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.652136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.652145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.652198 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.652380 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.652828 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.653656 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.654000 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.654056 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.654063 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.654264 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.654310 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:26.654418 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.652067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:33837 (local address 127.1.161.130:40143)
0923 22:08:26.652112 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.652114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.652225 (+ 111us) server_negotiation.cc:407] Connection header received
0923 22:08:26.652266 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.652267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.652276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.652343 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.652488 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.652885 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.653721 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.654021 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.654100 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.654108 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.654313 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.654346 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:26.654484 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.651988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:40143 (local address 127.1.161.131:33837)
0923 22:08:26.652168 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.652175 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.652217 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.652373 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.652374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.652375 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.652433 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.652434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.652977 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.652978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.653623 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.653625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.654136 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.654137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.654213 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.654237 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.654298 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.654332 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":40,"thread_start_us":83,"threads_started":1}
I20260923 22:08:26.654729 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11935054b0c64abcbd29860314849b21" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14" is_pre_election: true
I20260923 22:08:26.654778 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11935054b0c64abcbd29860314849b21" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055" is_pre_election: true
I20260923 22:08:26.654868 9309 raft_consensus.cc:2454] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:26.654887 9449 raft_consensus.cc:2454] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:26.655107 9521 leader_election.cc:302] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, e30528f598da45aeb45d84e48ab00055; no voters:
W20260923 22:08:26.655258 9356 tablet.cc:2369] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:26.655344 9668 raft_consensus.cc:2781] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:26.655390 9668 raft_consensus.cc:482] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:26.655411 9668 raft_consensus.cc:3037] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.656358 9668 raft_consensus.cc:504] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.656551 9668 leader_election.cc:288] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Requested vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:26.656635 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11935054b0c64abcbd29860314849b21" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055"
I20260923 22:08:26.656689 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11935054b0c64abcbd29860314849b21" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14"
I20260923 22:08:26.656715 9449 raft_consensus.cc:3037] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.656770 9309 raft_consensus.cc:3037] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.657426 9449 raft_consensus.cc:2454] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:26.657589 9521 leader_election.cc:302] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, e30528f598da45aeb45d84e48ab00055; no voters:
I20260923 22:08:26.657610 9309 raft_consensus.cc:2454] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:26.657681 9668 raft_consensus.cc:2781] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:26.657804 9668 raft_consensus.cc:686] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 1 LEADER]: Becoming Leader. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Running, Role: LEADER
I20260923 22:08:26.657891 9668 consensus_queue.cc:237] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } }
I20260923 22:08:26.658638 9152 catalog_manager.cc:5495] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c231298be9246f699b7efc300b21f02 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:26.668969 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.666973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.1.161.131:33011)
0923 22:08:26.667021 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.667023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.667055 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:08:26.667078 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.667078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.667092 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.667140 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:26.667319 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.667715 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.668390 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.668552 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.668597 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.668734 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:26.668825 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:26.668876 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.668918 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.668931 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:08:26.670397 9309 raft_consensus.cc:1261] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:26.670408 9449 raft_consensus.cc:1261] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:26.670627 9668 consensus_queue.cc:1035] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:26.670771 9670 consensus_queue.cc:1035] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:26.675465 9152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:26.675549 9152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:26.676249 9152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Sending DeleteTablet for 3 replicas of tablet 11935054b0c64abcbd29860314849b21
I20260923 22:08:26.676541 9569 tablet_service.cc:1514] Processing DeleteTablet for tablet 11935054b0c64abcbd29860314849b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:26 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35712
I20260923 22:08:26.676587 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 11935054b0c64abcbd29860314849b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:26 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:37886
I20260923 22:08:26.676834 9684 tablet_replica.cc:330] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: stopping tablet replica
I20260923 22:08:26.676875 9429 tablet_service.cc:1514] Processing DeleteTablet for tablet 11935054b0c64abcbd29860314849b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:26 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:50038
I20260923 22:08:26.677024 9683 tablet_replica.cc:330] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: stopping tablet replica
I20260923 22:08:26.677012 9152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:26.677186 9152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:26.677285 9683 raft_consensus.cc:2229] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:26.677412 9683 raft_consensus.cc:2258] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:26.676932 9684 raft_consensus.cc:2229] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:26.677798 9683 ts_tablet_manager.cc:1905] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:26.678381 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 6d24515398a2453cbdd28188811f90f2 (DEFAULT_TABLE table=test-table [id=ba470638a61649fdb37c645c51a5e30e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.678385 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 6d24515398a2453cbdd28188811f90f2 (DEFAULT_TABLE table=test-table [id=ba470638a61649fdb37c645c51a5e30e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.678491 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d24515398a2453cbdd28188811f90f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.678489 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d24515398a2453cbdd28188811f90f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.679185 9685 tablet_replica.cc:330] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: stopping tablet replica
I20260923 22:08:26.679291 9685 raft_consensus.cc:2229] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:26.679376 9685 raft_consensus.cc:2258] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:26.679776 9685 ts_tablet_manager.cc:1905] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:26.679832 9660 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Bootstrap starting.
I20260923 22:08:26.680064 9662 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Bootstrap starting.
I20260923 22:08:26.680199 9660 tablet_bootstrap.cc:654] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.680438 9662 tablet_bootstrap.cc:654] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.680796 9569 tablet_service.cc:1467] Processing CreateTablet for tablet 6d24515398a2453cbdd28188811f90f2 (DEFAULT_TABLE table=test-table [id=ba470638a61649fdb37c645c51a5e30e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:26.681010 9685 ts_tablet_manager.cc:1918] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:26.681022 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d24515398a2453cbdd28188811f90f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:26.681099 9685 log.cc:1196] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/11935054b0c64abcbd29860314849b21
I20260923 22:08:26.681335 9685 ts_tablet_manager.cc:1939] T 11935054b0c64abcbd29860314849b21 P e30528f598da45aeb45d84e48ab00055: Deleting consensus metadata
I20260923 22:08:26.681417 9660 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: No bootstrap required, opened a new log
I20260923 22:08:26.681458 9660 ts_tablet_manager.cc:1397] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:26.681599 9660 raft_consensus.cc:348] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.681661 9660 raft_consensus.cc:374] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.681695 9660 raft_consensus.cc:729] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.681752 9660 consensus_queue.cc:260] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.681871 9660 ts_tablet_manager.cc:1428] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.682181 9137 catalog_manager.cc:4841] TS e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143): tablet 11935054b0c64abcbd29860314849b21 (table test-table [id=b3868e748ce94d998ebb9a73adf0211a]) successfully deleted
I20260923 22:08:26.682348 9663 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Bootstrap starting.
I20260923 22:08:26.682618 9683 ts_tablet_manager.cc:1918] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:26.682695 9683 log.cc:1196] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/11935054b0c64abcbd29860314849b21
I20260923 22:08:26.682714 9663 tablet_bootstrap.cc:654] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:26.682962 9683 ts_tablet_manager.cc:1939] T 11935054b0c64abcbd29860314849b21 P 6c231298be9246f699b7efc300b21f02: Deleting consensus metadata
I20260923 22:08:26.683319 9663 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: No bootstrap required, opened a new log
I20260923 22:08:26.683332 9684 raft_consensus.cc:2258] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:26.683360 9663 ts_tablet_manager.cc:1397] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:26.683365 9138 catalog_manager.cc:4841] TS 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011): tablet 11935054b0c64abcbd29860314849b21 (table test-table [id=b3868e748ce94d998ebb9a73adf0211a]) successfully deleted
I20260923 22:08:26.683475 9663 raft_consensus.cc:348] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.683542 9663 raft_consensus.cc:374] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.683562 9663 raft_consensus.cc:729] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.683595 9663 consensus_queue.cc:260] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.683660 9663 ts_tablet_manager.cc:1428] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:26.683729 9684 ts_tablet_manager.cc:1905] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:26.684939 9684 ts_tablet_manager.cc:1918] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:26.685034 9684 log.cc:1196] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/11935054b0c64abcbd29860314849b21
I20260923 22:08:26.685459 9684 ts_tablet_manager.cc:1939] T 11935054b0c64abcbd29860314849b21 P d2372e9f613e489c81b94b43eeaa1d14: Deleting consensus metadata
I20260923 22:08:26.685832 9136 catalog_manager.cc:4841] TS d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425): tablet 11935054b0c64abcbd29860314849b21 (table test-table [id=b3868e748ce94d998ebb9a73adf0211a]) successfully deleted
I20260923 22:08:26.686028 9662 tablet_bootstrap.cc:492] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: No bootstrap required, opened a new log
I20260923 22:08:26.686079 9662 ts_tablet_manager.cc:1397] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 22:08:26.686190 9662 raft_consensus.cc:348] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.686233 9662 raft_consensus.cc:374] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:26.686249 9662 raft_consensus.cc:729] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30528f598da45aeb45d84e48ab00055, State: Initialized, Role: FOLLOWER
I20260923 22:08:26.686285 9662 consensus_queue.cc:260] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.686378 9662 ts_tablet_manager.cc:1428] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260923 22:08:26.693456 9636 tablet.cc:2369] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:08:26.938910 9665 raft_consensus.cc:482] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:26.938992 9665 raft_consensus.cc:504] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.939297 9665 leader_election.cc:288] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143), 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011)
I20260923 22:08:26.942672 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.939476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:58151 (local address 127.1.161.130:40143)
0923 22:08:26.939528 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.939530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.939620 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:08:26.939647 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.939647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.939656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.939711 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.939908 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.940540 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.941877 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.942338 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.942404 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.942411 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.942580 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.942620 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:08:26.942672 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.939547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:40787 (local address 127.1.161.131:33011)
0923 22:08:26.939589 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.939591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.939601 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:26.939646 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.939647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.939658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.939711 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:26.939907 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.940547 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.941877 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.942344 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.942404 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.942411 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.942580 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.942612 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:26.942708 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.939451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33011 (local address 127.1.161.129:40787)
0923 22:08:26.939538 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.939547 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.939596 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.939732 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.939732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.939734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.939834 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.939835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.940617 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.940617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.941790 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.941793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.942418 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.942420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.942498 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.942505 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.942555 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.942604 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":40,"thread_start_us":44,"threads_started":1}
I20260923 22:08:26.942708 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.939424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:40143 (local address 127.1.161.129:58151)
0923 22:08:26.939532 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.939542 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.939598 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.939731 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.939731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.939734 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:26.939834 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.939835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.940616 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.940617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.941790 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.941793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.942441 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.942442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.942498 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.942506 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.942555 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.942604 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":109,"thread_start_us":52,"threads_started":1}
I20260923 22:08:26.942953 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d24515398a2453cbdd28188811f90f2" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c231298be9246f699b7efc300b21f02" is_pre_election: true
I20260923 22:08:26.943044 9589 raft_consensus.cc:2454] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 0.
I20260923 22:08:26.943251 9242 leader_election.cc:302] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, d2372e9f613e489c81b94b43eeaa1d14; no voters:
I20260923 22:08:26.943379 9665 raft_consensus.cc:2781] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:26.943414 9665 raft_consensus.cc:482] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:26.943436 9665 raft_consensus.cc:3037] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.943403 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d24515398a2453cbdd28188811f90f2" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055" is_pre_election: true
I20260923 22:08:26.943476 9449 raft_consensus.cc:2454] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 0.
I20260923 22:08:26.944085 9665 raft_consensus.cc:504] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.944216 9665 leader_election.cc:288] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 election: Requested vote from peers e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143), 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011)
I20260923 22:08:26.944362 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d24515398a2453cbdd28188811f90f2" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055"
I20260923 22:08:26.944381 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d24515398a2453cbdd28188811f90f2" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c231298be9246f699b7efc300b21f02"
I20260923 22:08:26.944434 9589 raft_consensus.cc:3037] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.944434 9449 raft_consensus.cc:3037] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:26.945124 9449 raft_consensus.cc:2454] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 1.
I20260923 22:08:26.945124 9589 raft_consensus.cc:2454] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 1.
I20260923 22:08:26.945329 9241 leader_election.cc:302] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2372e9f613e489c81b94b43eeaa1d14, e30528f598da45aeb45d84e48ab00055; no voters:
I20260923 22:08:26.945456 9665 raft_consensus.cc:2781] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:26.945616 9665 raft_consensus.cc:686] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 LEADER]: Becoming Leader. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Running, Role: LEADER
I20260923 22:08:26.945691 9665 consensus_queue.cc:237] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:26.946405 9151 catalog_manager.cc:5495] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "d2372e9f613e489c81b94b43eeaa1d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } health_report { overall_health: UNKNOWN } } }
Sep 23 22:08:26 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201305, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.190@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:26.997522 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.983725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:46873 (local address 127.0.0.1:34964)
0923 22:08:26.983956 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:26.983966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:26.984043 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:26.984205 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:26.984206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:26.984491 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:26.984542 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.984544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.985176 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.985177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.985688 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:26.985689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.985997 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.985998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:26.986042 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.993375 (+ 7333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:26.993379 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:26.994827 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:26.996551 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.996557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.996559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.996770 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.996987 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:26.996988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:26.996988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:26.997047 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:26.997258 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:26.997264 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:26.997315 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 22:08:26.997381 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:08:26.997465 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":151,"threads_started":1}
I20260923 22:08:26.997577 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:26.983826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.1.161.190:46873)
0923 22:08:26.983950 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:26.983952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:26.984027 (+ 75us) server_negotiation.cc:407] Connection header received
0923 22:08:26.984095 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:26.984096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:26.984106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:26.984192 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:26.984625 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.985083 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.985769 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:26.985927 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:26.985991 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:26.994907 (+ 8916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:26.994908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:26.994908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:26.994914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:26.996468 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.996853 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.996854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.996855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.996919 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:26.997106 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:26.997107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:26.997107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:26.997200 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:26.997249 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:26.997407 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:08:26.997506 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260923 22:08:26.999130 9151 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:34964:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 22:08:26.999353 9151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:27.000576 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 2a71820536514c34bd8446b15f2f7ade (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd330e52e964c02a36b0b0ae4d6ceda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:27.000581 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 2a71820536514c34bd8446b15f2f7ade (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd330e52e964c02a36b0b0ae4d6ceda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:27.000705 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a71820536514c34bd8446b15f2f7ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.000705 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a71820536514c34bd8446b15f2f7ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.001981 9662 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: Bootstrap starting.
I20260923 22:08:27.001976 9660 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: Bootstrap starting.
I20260923 22:08:27.002044 9569 tablet_service.cc:1467] Processing CreateTablet for tablet 2a71820536514c34bd8446b15f2f7ade (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddd330e52e964c02a36b0b0ae4d6ceda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 22:08:27.002170 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a71820536514c34bd8446b15f2f7ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.002336 9662 tablet_bootstrap.cc:654] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.002391 9660 tablet_bootstrap.cc:654] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.002923 9662 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: No bootstrap required, opened a new log
I20260923 22:08:27.002971 9662 ts_tablet_manager.cc:1397] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:27.003091 9662 raft_consensus.cc:348] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.003192 9662 raft_consensus.cc:374] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.003221 9662 raft_consensus.cc:729] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30528f598da45aeb45d84e48ab00055, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.003266 9662 consensus_queue.cc:260] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.003381 9662 ts_tablet_manager.cc:1428] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:27.003572 9663 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: Bootstrap starting.
I20260923 22:08:27.003603 9667 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.003686 9667 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.003953 9663 tablet_bootstrap.cc:654] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.004319 9660 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: No bootstrap required, opened a new log
I20260923 22:08:27.004372 9660 ts_tablet_manager.cc:1397] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:27.004505 9660 raft_consensus.cc:348] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.004573 9660 raft_consensus.cc:374] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.004611 9660 raft_consensus.cc:729] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.004662 9660 consensus_queue.cc:260] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.004782 9660 ts_tablet_manager.cc:1428] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.004832 9665 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.004940 9665 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.005185 9663 tablet_bootstrap.cc:492] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: No bootstrap required, opened a new log
I20260923 22:08:27.005249 9663 ts_tablet_manager.cc:1397] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:27.005366 9663 raft_consensus.cc:348] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.005439 9663 raft_consensus.cc:374] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.005467 9663 raft_consensus.cc:729] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.005512 9663 consensus_queue.cc:260] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.005604 9663 ts_tablet_manager.cc:1428] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.005658 9679 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.005762 9679 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.266578 9679 raft_consensus.cc:482] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:27.266671 9679 raft_consensus.cc:504] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.266824 9679 leader_election.cc:288] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:27.267012 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a71820536514c34bd8446b15f2f7ade" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14" is_pre_election: true
I20260923 22:08:27.267053 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a71820536514c34bd8446b15f2f7ade" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055" is_pre_election: true
I20260923 22:08:27.267125 9309 raft_consensus.cc:2454] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:27.267155 9449 raft_consensus.cc:2454] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:27.267313 9520 leader_election.cc:302] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, d2372e9f613e489c81b94b43eeaa1d14; no voters:
I20260923 22:08:27.267448 9679 raft_consensus.cc:2781] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:27.267493 9679 raft_consensus.cc:482] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:27.267519 9679 raft_consensus.cc:3037] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.268141 9679 raft_consensus.cc:504] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.268271 9679 leader_election.cc:288] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Requested vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:27.268401 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a71820536514c34bd8446b15f2f7ade" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14"
I20260923 22:08:27.268404 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a71820536514c34bd8446b15f2f7ade" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055"
I20260923 22:08:27.268466 9309 raft_consensus.cc:3037] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.268489 9449 raft_consensus.cc:3037] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.269019 9449 raft_consensus.cc:2454] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:27.269022 9309 raft_consensus.cc:2454] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:27.269191 9520 leader_election.cc:302] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, d2372e9f613e489c81b94b43eeaa1d14; no voters:
I20260923 22:08:27.269263 9679 raft_consensus.cc:2781] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:27.269352 9679 raft_consensus.cc:686] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [term 1 LEADER]: Becoming Leader. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Running, Role: LEADER
I20260923 22:08:27.269394 9679 consensus_queue.cc:237] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.269624 9703 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: TxnStatusTablet state changed. Reason: New leader 6c231298be9246f699b7efc300b21f02. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.269708 9703 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:27.269896 9705 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 22:08:27.270015 9705 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 22:08:27.269954 9152 catalog_manager.cc:5495] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c231298be9246f699b7efc300b21f02 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:27.285099 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.282648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33011 (local address 127.0.0.1:54846)
0923 22:08:27.282767 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.282777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.282831 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.282971 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.282972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.282973 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:27.283018 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.283021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.283603 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.283604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.284421 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.284423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.284703 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.284704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.284764 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.284779 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.284925 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.284931 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:08:27.284972 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 22:08:27.285009 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260923 22:08:27.285110 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.282703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.1.161.131:33011)
0923 22:08:27.282743 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.282745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.282811 (+ 66us) server_negotiation.cc:407] Connection header received
0923 22:08:27.282901 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.282902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.282909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.282947 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.283070 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.283540 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.284487 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.284638 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.284682 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.284852 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.284874 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.284908 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.284981 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.284996 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:27.285972 9449 raft_consensus.cc:1261] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.286000 9309 raft_consensus.cc:1261] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.286186 9679 consensus_queue.cc:1035] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.286315 9703 consensus_queue.cc:1035] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.286770 9665 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: TxnStatusTablet state changed. Reason: New leader 6c231298be9246f699b7efc300b21f02. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.286854 9665 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.286826 9667 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: TxnStatusTablet state changed. Reason: New leader 6c231298be9246f699b7efc300b21f02. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.286906 9667 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.287496 9708 mvcc.cc:205] Tried to move back new op lower bound from 7332664554641641472 to 7332664554575544320. Current Snapshot: MvccSnapshot[applied={T|T < 7332664554641641472}]
I20260923 22:08:27.287593 9679 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.287660 9679 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:27.287763 9667 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.287835 9667 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P e30528f598da45aeb45d84e48ab00055: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.287979 9679 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.288049 9679 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P 6c231298be9246f699b7efc300b21f02: This TxnStatusTablet replica's current role is: LEADER
I20260923 22:08:27.288291 9665 tablet_replica.cc:439] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } } }
I20260923 22:08:27.288362 9665 tablet_replica.cc:442] T 2a71820536514c34bd8446b15f2f7ade P d2372e9f613e489c81b94b43eeaa1d14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 22:08:27.288666 9711 mvcc.cc:205] Tried to move back new op lower bound from 7332664554641641472 to 7332664554575544320. Current Snapshot: MvccSnapshot[applied={T|T < 7332664554641641472}]
I20260923 22:08:27.293761 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.289938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45776 (local address 127.1.161.129:38425)
0923 22:08:27.290020 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.290022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.291424 (+ 1402us) server_negotiation.cc:407] Connection header received
0923 22:08:27.291458 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.291458 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.291476 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.291551 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.291693 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.292325 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.292946 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.293206 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.293267 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.293469 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.293576 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.293639 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.293671 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.293687 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"thread_start_us":47,"threads_started":1}
I20260923 22:08:27.298355 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.296028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:46225 (local address 127.1.161.131:33011)
0923 22:08:27.296065 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.296067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.296280 (+ 213us) server_negotiation.cc:407] Connection header received
0923 22:08:27.296303 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.296304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.296313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.296365 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.296478 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.296885 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.297865 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.298007 (+ 142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.298051 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.298212 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.298231 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.298267 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.298305 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.298318 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:08:27.298405 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.295993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33011 (local address 127.1.161.129:46225)
0923 22:08:27.296147 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.296158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.296237 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.296385 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.296386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.296387 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:27.296424 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.296425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.296954 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.296955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.297792 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.297794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.298072 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.298073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.298127 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.298140 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.298263 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.298264 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:27.298304 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:08:27.298342 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260923 22:08:27.300465 9449 raft_consensus.cc:1261] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Refusing update from remote peer d2372e9f613e489c81b94b43eeaa1d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.300531 9589 raft_consensus.cc:1261] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Refusing update from remote peer d2372e9f613e489c81b94b43eeaa1d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.300642 9665 consensus_queue.cc:1035] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.300732 9691 consensus_queue.cc:1035] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.302129 9709 mvcc.cc:205] Tried to move back new op lower bound from 7332664554701344768 to 7332664553249951744. Current Snapshot: MvccSnapshot[applied={T|T < 7332664554701344768}]
I20260923 22:08:27.307752 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.305320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60241 (local address 127.1.161.129:38425)
0923 22:08:27.305349 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.305351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.305458 (+ 107us) server_negotiation.cc:407] Connection header received
0923 22:08:27.305514 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.305515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.305521 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.305559 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.305640 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.305997 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.307120 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.307375 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.307451 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.307608 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.307628 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.307662 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.307704 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.307716 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:08:27.307807 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.305278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:38425 (local address 127.1.161.131:60241)
0923 22:08:27.305415 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.305423 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.305468 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.305561 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.305562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.305564 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:27.305596 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.305597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.306060 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.306061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.306974 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.306976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.307466 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.307467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.307520 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.307532 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.307661 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.307662 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:27.307702 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:08:27.307736 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260923 22:08:27.316946 9152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.317029 9152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.317618 9152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Sending DeleteTablet for 3 replicas of tablet 6d24515398a2453cbdd28188811f90f2
I20260923 22:08:27.317812 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d24515398a2453cbdd28188811f90f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:37886
I20260923 22:08:27.317970 9429 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d24515398a2453cbdd28188811f90f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:50038
I20260923 22:08:27.320595 9569 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d24515398a2453cbdd28188811f90f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35712
I20260923 22:08:27.320659 9730 tablet_replica.cc:330] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: stopping tablet replica
I20260923 22:08:27.320711 9730 raft_consensus.cc:2229] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:27.320777 9730 raft_consensus.cc:2258] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.320791 9733 tablet_replica.cc:330] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: stopping tablet replica
I20260923 22:08:27.320863 9731 tablet_replica.cc:330] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: stopping tablet replica
I20260923 22:08:27.320912 9731 raft_consensus.cc:2229] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.320945 9731 raft_consensus.cc:2258] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.320966 9733 raft_consensus.cc:2229] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.321063 9733 raft_consensus.cc:2258] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.321134 9731 ts_tablet_manager.cc:1905] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.321350 9733 ts_tablet_manager.cc:1905] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.321376 9730 ts_tablet_manager.cc:1905] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.322645 9731 ts_tablet_manager.cc:1918] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:27.322703 9731 log.cc:1196] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/6d24515398a2453cbdd28188811f90f2
I20260923 22:08:27.322925 9731 ts_tablet_manager.cc:1939] T 6d24515398a2453cbdd28188811f90f2 P e30528f598da45aeb45d84e48ab00055: Deleting consensus metadata
I20260923 22:08:27.323098 9733 ts_tablet_manager.cc:1918] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:27.323208 9733 log.cc:1196] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/6d24515398a2453cbdd28188811f90f2
I20260923 22:08:27.323258 9137 catalog_manager.cc:4841] TS e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143): tablet 6d24515398a2453cbdd28188811f90f2 (table test-table [id=ba470638a61649fdb37c645c51a5e30e]) successfully deleted
I20260923 22:08:27.323495 9733 ts_tablet_manager.cc:1939] T 6d24515398a2453cbdd28188811f90f2 P 6c231298be9246f699b7efc300b21f02: Deleting consensus metadata
I20260923 22:08:27.323856 9138 catalog_manager.cc:4841] TS 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011): tablet 6d24515398a2453cbdd28188811f90f2 (table test-table [id=ba470638a61649fdb37c645c51a5e30e]) successfully deleted
I20260923 22:08:27.327350 9730 ts_tablet_manager.cc:1918] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:27.327406 9730 log.cc:1196] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/6d24515398a2453cbdd28188811f90f2
I20260923 22:08:27.327618 9730 ts_tablet_manager.cc:1939] T 6d24515398a2453cbdd28188811f90f2 P d2372e9f613e489c81b94b43eeaa1d14: Deleting consensus metadata
I20260923 22:08:27.327865 9136 catalog_manager.cc:4841] TS d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425): tablet 6d24515398a2453cbdd28188811f90f2 (table test-table [id=ba470638a61649fdb37c645c51a5e30e]) successfully deleted
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:27.357141 9699 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:27.350814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.1.161.190:46873)
0923 22:08:27.350854 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.350856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.350996 (+ 140us) server_negotiation.cc:407] Connection header received
0923 22:08:27.351044 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.351045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.351058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.351114 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:27.351769 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.352449 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.353406 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.353669 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.353739 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.357062 (+ 3323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34972: BlockingRecv error: recv got EOF from 127.0.0.1:34972 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM
I20260923 22:08:27.365585 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.358043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.1.161.190:46873)
0923 22:08:27.358084 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.358085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.358207 (+ 122us) server_negotiation.cc:407] Connection header received
0923 22:08:27.358244 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.358245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.358253 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.358312 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:27.358814 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.359538 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.360432 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.360711 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.360774 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.363023 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:27.363024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:27.363024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:27.363031 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:27.364576 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:27.364912 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:27.364913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:27.364914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:27.364965 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:27.365147 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:27.365148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:27.365148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:27.365251 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:27.365362 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.365443 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.365513 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 22:08:27.367087 9152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:27.367276 9152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:27.368562 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 331fcdb2617843b4a98e748cfb60db87 (DEFAULT_TABLE table=test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.368561 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 331fcdb2617843b4a98e748cfb60db87 (DEFAULT_TABLE table=test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.368675 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331fcdb2617843b4a98e748cfb60db87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.368676 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331fcdb2617843b4a98e748cfb60db87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.369858 9569 tablet_service.cc:1467] Processing CreateTablet for tablet 331fcdb2617843b4a98e748cfb60db87 (DEFAULT_TABLE table=test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.369962 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 331fcdb2617843b4a98e748cfb60db87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.371152 9663 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Bootstrap starting.
I20260923 22:08:27.371232 9662 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Bootstrap starting.
I20260923 22:08:27.371560 9663 tablet_bootstrap.cc:654] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.371584 9662 tablet_bootstrap.cc:654] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.371156 9660 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Bootstrap starting.
I20260923 22:08:27.372677 9660 tablet_bootstrap.cc:654] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.373335 9660 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: No bootstrap required, opened a new log
I20260923 22:08:27.373382 9660 ts_tablet_manager.cc:1397] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:27.373482 9662 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: No bootstrap required, opened a new log
I20260923 22:08:27.373519 9662 ts_tablet_manager.cc:1397] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:27.373515 9660 raft_consensus.cc:348] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.373570 9660 raft_consensus.cc:374] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.373588 9660 raft_consensus.cc:729] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.373621 9660 consensus_queue.cc:260] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.373634 9662 raft_consensus.cc:348] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.373689 9662 raft_consensus.cc:374] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.373690 9660 ts_tablet_manager.cc:1428] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.373705 9662 raft_consensus.cc:729] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30528f598da45aeb45d84e48ab00055, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.373741 9662 consensus_queue.cc:260] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.373808 9662 ts_tablet_manager.cc:1428] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.374449 9663 tablet_bootstrap.cc:492] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: No bootstrap required, opened a new log
I20260923 22:08:27.374495 9663 ts_tablet_manager.cc:1397] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 22:08:27.374600 9663 raft_consensus.cc:348] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.374658 9663 raft_consensus.cc:374] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.374680 9663 raft_consensus.cc:729] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.374717 9663 consensus_queue.cc:260] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.374790 9663 ts_tablet_manager.cc:1428] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.556481 9726 raft_consensus.cc:482] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:27.556600 9726 raft_consensus.cc:504] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.556824 9726 leader_election.cc:288] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143), 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011)
I20260923 22:08:27.556983 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331fcdb2617843b4a98e748cfb60db87" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055" is_pre_election: true
I20260923 22:08:27.557080 9449 raft_consensus.cc:2454] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 0.
I20260923 22:08:27.557024 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331fcdb2617843b4a98e748cfb60db87" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c231298be9246f699b7efc300b21f02" is_pre_election: true
I20260923 22:08:27.557126 9589 raft_consensus.cc:2454] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 0.
I20260923 22:08:27.557235 9241 leader_election.cc:302] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2372e9f613e489c81b94b43eeaa1d14, e30528f598da45aeb45d84e48ab00055; no voters:
I20260923 22:08:27.557330 9726 raft_consensus.cc:2781] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:27.557399 9726 raft_consensus.cc:482] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:27.557463 9726 raft_consensus.cc:3037] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.558236 9726 raft_consensus.cc:504] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.558359 9726 leader_election.cc:288] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 election: Requested vote from peers e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143), 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011)
I20260923 22:08:27.558501 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331fcdb2617843b4a98e748cfb60db87" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055"
I20260923 22:08:27.558511 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "331fcdb2617843b4a98e748cfb60db87" candidate_uuid: "d2372e9f613e489c81b94b43eeaa1d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c231298be9246f699b7efc300b21f02"
I20260923 22:08:27.558570 9589 raft_consensus.cc:3037] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.558570 9449 raft_consensus.cc:3037] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.559238 9449 raft_consensus.cc:2454] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 1.
I20260923 22:08:27.559286 9589 raft_consensus.cc:2454] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2372e9f613e489c81b94b43eeaa1d14 in term 1.
I20260923 22:08:27.559379 9241 leader_election.cc:302] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2372e9f613e489c81b94b43eeaa1d14, e30528f598da45aeb45d84e48ab00055; no voters:
I20260923 22:08:27.559461 9726 raft_consensus.cc:2781] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:27.559512 9726 raft_consensus.cc:686] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 LEADER]: Becoming Leader. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Running, Role: LEADER
I20260923 22:08:27.559552 9726 consensus_queue.cc:237] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.560112 9151 catalog_manager.cc:5495] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "d2372e9f613e489c81b94b43eeaa1d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } health_report { overall_health: UNKNOWN } } }
I20260923 22:08:27.598398 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.596112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.1.161.129:38425)
0923 22:08:27.596154 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.596156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.596165 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:27.596233 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.596234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.596243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.596354 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.596489 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.596959 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.597731 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.597921 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.598007 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.598159 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.598193 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.598248 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.598306 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.598323 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:27.599085 9589 raft_consensus.cc:1261] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Refusing update from remote peer d2372e9f613e489c81b94b43eeaa1d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.599085 9449 raft_consensus.cc:1261] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Refusing update from remote peer d2372e9f613e489c81b94b43eeaa1d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.599359 9691 consensus_queue.cc:1035] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.599488 9726 consensus_queue.cc:1035] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.601811 9151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.601889 9151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.602389 9151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Sending DeleteTablet for 3 replicas of tablet 331fcdb2617843b4a98e748cfb60db87
I20260923 22:08:27.602591 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 331fcdb2617843b4a98e748cfb60db87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:37886
I20260923 22:08:27.602612 9569 tablet_service.cc:1514] Processing DeleteTablet for tablet 331fcdb2617843b4a98e748cfb60db87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35712
I20260923 22:08:27.602662 9730 tablet_replica.cc:330] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: stopping tablet replica
I20260923 22:08:27.602640 9429 tablet_service.cc:1514] Processing DeleteTablet for tablet 331fcdb2617843b4a98e748cfb60db87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:50038
I20260923 22:08:27.602703 9730 raft_consensus.cc:2229] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:27.602696 9733 tablet_replica.cc:330] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: stopping tablet replica
I20260923 22:08:27.602725 9731 tablet_replica.cc:330] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: stopping tablet replica
I20260923 22:08:27.602743 9733 raft_consensus.cc:2229] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.602747 9730 raft_consensus.cc:2258] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.602764 9731 raft_consensus.cc:2229] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.602795 9733 raft_consensus.cc:2258] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.602825 9731 raft_consensus.cc:2258] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.603044 9730 ts_tablet_manager.cc:1905] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.603204 9733 ts_tablet_manager.cc:1905] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.603257 9151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:27.603412 9151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:27.604485 9730 ts_tablet_manager.cc:1918] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:27.604507 9289 tablet_service.cc:1467] Processing CreateTablet for tablet 580725f2471b4342ac196490c3ce1e1e (DEFAULT_TABLE table=test-table [id=f818a5f5384847458f454d48f716e08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.604552 9730 log.cc:1196] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/331fcdb2617843b4a98e748cfb60db87
I20260923 22:08:27.604616 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 580725f2471b4342ac196490c3ce1e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.604821 9730 ts_tablet_manager.cc:1939] T 331fcdb2617843b4a98e748cfb60db87 P d2372e9f613e489c81b94b43eeaa1d14: Deleting consensus metadata
I20260923 22:08:27.604861 9731 ts_tablet_manager.cc:1905] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.605201 9136 catalog_manager.cc:4841] TS d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425): tablet 331fcdb2617843b4a98e748cfb60db87 (table test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]) successfully deleted
I20260923 22:08:27.605259 9569 tablet_service.cc:1467] Processing CreateTablet for tablet 580725f2471b4342ac196490c3ce1e1e (DEFAULT_TABLE table=test-table [id=f818a5f5384847458f454d48f716e08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.605878 9660 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: Bootstrap starting.
I20260923 22:08:27.605959 9429 tablet_service.cc:1467] Processing CreateTablet for tablet 580725f2471b4342ac196490c3ce1e1e (DEFAULT_TABLE table=test-table [id=f818a5f5384847458f454d48f716e08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:27.606026 9733 ts_tablet_manager.cc:1918] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:27.606050 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 580725f2471b4342ac196490c3ce1e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.606073 9733 log.cc:1196] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/331fcdb2617843b4a98e748cfb60db87
I20260923 22:08:27.606159 9660 tablet_bootstrap.cc:654] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.606341 9733 ts_tablet_manager.cc:1939] T 331fcdb2617843b4a98e748cfb60db87 P 6c231298be9246f699b7efc300b21f02: Deleting consensus metadata
I20260923 22:08:27.606369 9731 ts_tablet_manager.cc:1918] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:27.606415 9731 log.cc:1196] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/331fcdb2617843b4a98e748cfb60db87
I20260923 22:08:27.606628 9731 ts_tablet_manager.cc:1939] T 331fcdb2617843b4a98e748cfb60db87 P e30528f598da45aeb45d84e48ab00055: Deleting consensus metadata
I20260923 22:08:27.606796 9660 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: No bootstrap required, opened a new log
I20260923 22:08:27.606848 9660 ts_tablet_manager.cc:1397] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:27.607003 9660 raft_consensus.cc:348] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.607107 9660 raft_consensus.cc:374] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.607139 9660 raft_consensus.cc:729] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2372e9f613e489c81b94b43eeaa1d14, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.607154 9137 catalog_manager.cc:4841] TS e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143): tablet 331fcdb2617843b4a98e748cfb60db87 (table test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]) successfully deleted
I20260923 22:08:27.607213 9660 consensus_queue.cc:260] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.607306 9660 ts_tablet_manager.cc:1428] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:27.607324 9662 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Bootstrap starting.
I20260923 22:08:27.607966 9662 tablet_bootstrap.cc:654] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.608276 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 580725f2471b4342ac196490c3ce1e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:27.608462 9138 catalog_manager.cc:4841] TS 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011): tablet 331fcdb2617843b4a98e748cfb60db87 (table test-table [id=b523a9cdee1340f6a5eafaf8f7bd2900]) successfully deleted
I20260923 22:08:27.609124 9663 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Bootstrap starting.
I20260923 22:08:27.609302 9662 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: No bootstrap required, opened a new log
I20260923 22:08:27.609350 9662 ts_tablet_manager.cc:1397] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:27.609395 9663 tablet_bootstrap.cc:654] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:27.609485 9662 raft_consensus.cc:348] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.609570 9662 raft_consensus.cc:374] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.609596 9662 raft_consensus.cc:729] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30528f598da45aeb45d84e48ab00055, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.609629 9662 consensus_queue.cc:260] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.609705 9662 ts_tablet_manager.cc:1428] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.609870 9663 tablet_bootstrap.cc:492] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: No bootstrap required, opened a new log
I20260923 22:08:27.609912 9663 ts_tablet_manager.cc:1397] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:08:27.610002 9663 raft_consensus.cc:348] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.610075 9663 raft_consensus.cc:374] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:27.610105 9663 raft_consensus.cc:729] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Initialized, Role: FOLLOWER
I20260923 22:08:27.610141 9663 consensus_queue.cc:260] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.610244 9663 ts_tablet_manager.cc:1428] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:27.706547 9728 raft_consensus.cc:482] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:27.706620 9728 raft_consensus.cc:504] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.706825 9728 leader_election.cc:288] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:27.707010 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "580725f2471b4342ac196490c3ce1e1e" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14" is_pre_election: true
I20260923 22:08:27.707010 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "580725f2471b4342ac196490c3ce1e1e" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055" is_pre_election: true
I20260923 22:08:27.707103 9309 raft_consensus.cc:2454] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:27.707103 9449 raft_consensus.cc:2454] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 0.
I20260923 22:08:27.707289 9520 leader_election.cc:302] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, d2372e9f613e489c81b94b43eeaa1d14; no voters:
I20260923 22:08:27.707427 9728 raft_consensus.cc:2781] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:27.707466 9728 raft_consensus.cc:482] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:27.707517 9728 raft_consensus.cc:3037] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.708128 9728 raft_consensus.cc:504] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.708256 9728 leader_election.cc:288] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Requested vote from peers d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425), e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143)
I20260923 22:08:27.708406 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "580725f2471b4342ac196490c3ce1e1e" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2372e9f613e489c81b94b43eeaa1d14"
I20260923 22:08:27.708477 9309 raft_consensus.cc:3037] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.708424 9449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "580725f2471b4342ac196490c3ce1e1e" candidate_uuid: "6c231298be9246f699b7efc300b21f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e30528f598da45aeb45d84e48ab00055"
I20260923 22:08:27.708508 9449 raft_consensus.cc:3037] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:27.708992 9309 raft_consensus.cc:2454] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:27.709079 9449 raft_consensus.cc:2454] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c231298be9246f699b7efc300b21f02 in term 1.
I20260923 22:08:27.709137 9520 leader_election.cc:302] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c231298be9246f699b7efc300b21f02, d2372e9f613e489c81b94b43eeaa1d14; no voters:
I20260923 22:08:27.709204 9728 raft_consensus.cc:2781] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:27.709241 9728 raft_consensus.cc:686] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 1 LEADER]: Becoming Leader. State: Replica: 6c231298be9246f699b7efc300b21f02, State: Running, Role: LEADER
I20260923 22:08:27.709283 9728 consensus_queue.cc:237] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } }
I20260923 22:08:27.709786 9152 catalog_manager.cc:5495] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c231298be9246f699b7efc300b21f02 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "6c231298be9246f699b7efc300b21f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c231298be9246f699b7efc300b21f02" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33011 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:27.725769 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.723920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54850 (local address 127.1.161.131:33011)
0923 22:08:27.723953 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.723954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.723971 (+ 17us) server_negotiation.cc:407] Connection header received
0923 22:08:27.723988 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.723989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.723997 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.724035 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.724118 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.724537 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.725174 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.725353 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.725421 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.725547 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.725582 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.725639 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.725691 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.725707 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:08:27.728988 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.727407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:42247 (local address 127.1.161.131:33011)
0923 22:08:27.727440 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:27.727441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:27.727449 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:27.727499 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:27.727500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:27.727506 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:27.727565 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:27.727639 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.727995 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.728533 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.728668 (+ 135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.728709 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.728819 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.728851 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.728878 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:27.728915 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:08:27.728924 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:08:27.729022 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.727337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33011 (local address 127.1.161.131:42247)
0923 22:08:27.727395 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.727404 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:27.727439 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:27.727568 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:27.727569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:27.727569 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 22:08:27.727610 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.727611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.728046 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.728046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.728475 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:27.728476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:27.728724 (+ 248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:27.728725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:27.728773 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:27.728777 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 22:08:27.728882 (+ 105us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 22:08:27.728883 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:27.728907 (+ 24us) client_negotiation.cc:240] Negotiation successful
0923 22:08:27.728945 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 22:08:27.730440 9449 raft_consensus.cc:1261] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.730461 9309 raft_consensus.cc:1261] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Refusing update from remote peer 6c231298be9246f699b7efc300b21f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:27.730609 9728 consensus_queue.cc:1035] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2372e9f613e489c81b94b43eeaa1d14" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.730763 9703 consensus_queue.cc:1035] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e30528f598da45aeb45d84e48ab00055" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:27.739298 9151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.739359 9151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:27.739782 9151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b02f7819601d46ebb6fc0dc3ac40565c: Sending DeleteTablet for 3 replicas of tablet 580725f2471b4342ac196490c3ce1e1e
I20260923 22:08:27.739990 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet 580725f2471b4342ac196490c3ce1e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:37886
I20260923 22:08:27.739987 9429 tablet_service.cc:1514] Processing DeleteTablet for tablet 580725f2471b4342ac196490c3ce1e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:50038
I20260923 22:08:27.740072 9731 tablet_replica.cc:330] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: stopping tablet replica
I20260923 22:08:27.740070 9569 tablet_service.cc:1514] Processing DeleteTablet for tablet 580725f2471b4342ac196490c3ce1e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:27 UTC) from {username='oryx', principal='oryx/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:35712
I20260923 22:08:27.740123 9731 raft_consensus.cc:2229] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.740131 9733 tablet_replica.cc:330] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: stopping tablet replica
I20260923 22:08:27.740157 9731 raft_consensus.cc:2258] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.740175 9733 raft_consensus.cc:2229] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:27.740224 9733 raft_consensus.cc:2258] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.740422 9731 ts_tablet_manager.cc:1905] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.740446 9730 tablet_replica.cc:330] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: stopping tablet replica
I20260923 22:08:27.740499 9730 raft_consensus.cc:2229] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:27.740494 9733 ts_tablet_manager.cc:1905] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.740535 9730 raft_consensus.cc:2258] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:27.740770 9730 ts_tablet_manager.cc:1905] T 580725f2471b4342ac196490c3ce1e1e P d2372e9f613e489c81b94b43eeaa1d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:27.741874 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9224
I20260923 22:08:27.742575 9731 ts_tablet_manager.cc:1918] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:27.742636 9731 log.cc:1196] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/580725f2471b4342ac196490c3ce1e1e
I20260923 22:08:27.742815 9731 ts_tablet_manager.cc:1939] T 580725f2471b4342ac196490c3ce1e1e P e30528f598da45aeb45d84e48ab00055: Deleting consensus metadata
I20260923 22:08:27.743106 9137 catalog_manager.cc:4841] TS e30528f598da45aeb45d84e48ab00055 (127.1.161.130:40143): tablet 580725f2471b4342ac196490c3ce1e1e (table test-table [id=f818a5f5384847458f454d48f716e08c]) successfully deleted
I20260923 22:08:27.747293 9733 ts_tablet_manager.cc:1918] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 22:08:27.747362 9733 log.cc:1196] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/580725f2471b4342ac196490c3ce1e1e
I20260923 22:08:27.747586 9733 ts_tablet_manager.cc:1939] T 580725f2471b4342ac196490c3ce1e1e P 6c231298be9246f699b7efc300b21f02: Deleting consensus metadata
I20260923 22:08:27.747876 9138 catalog_manager.cc:4841] TS 6c231298be9246f699b7efc300b21f02 (127.1.161.131:33011): tablet 580725f2471b4342ac196490c3ce1e1e (table test-table [id=f818a5f5384847458f454d48f716e08c]) successfully deleted
W20260923 22:08:27.748725 9136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:38425 (error 108)
I20260923 22:08:27.748852 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9364
I20260923 22:08:27.749926 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:27.749763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:38425 (local address 127.0.0.1:45794)
0923 22:08:27.749857 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:27.749895 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:38425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":55,"threads_started":1}
W20260923 22:08:27.750054 9136 catalog_manager.cc:4584] TS d2372e9f613e489c81b94b43eeaa1d14 (127.1.161.129:38425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 580725f2471b4342ac196490c3ce1e1e: Network error: Client connection negotiation failed: client connection to 127.1.161.129:38425: connect: Connection refused (error 111)
I20260923 22:08:27.753310 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9504
I20260923 22:08:27.758186 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9120
2026-09-23T22:08:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2883 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9763](info): set up 2 sockets
Sep 23 22:08:27 dist-test-slave-npvd krb5kdc[9763](info): commencing operation
krb5kdc: starting...
W20260923 22:08:28.683452 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.905s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:28 dist-test-slave-npvd krb5kdc[9763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:28.788069 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46555
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:46555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:28.857915 9776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:28.858110 9776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:28.858142 9776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:28.859100 9776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:28.859165 9776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:28.859202 9776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:28.859229 9776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:28.859246 9776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:28.860559 9776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:46555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:28.860785 9776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:28.860977 9776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:28.864952 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:28.865020 9784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:28.864964 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:28.865409 9776 server_base.cc:1034] running on GCE node
I20260923 22:08:28.865643 9776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:28.865880 9776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:28.867048 9776 hybrid_clock.cc:648] HybridClock initialized: now 1790201308867042 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T22:08:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1115 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:28 dist-test-slave-npvd krb5kdc[9789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:28 dist-test-slave-npvd krb5kdc[9789](info): set up 2 sockets
Sep 23 22:08:28 dist-test-slave-npvd krb5kdc[9789](info): commencing operation
krb5kdc: starting...
W20260923 22:08:29.799546 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.905s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:29 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:29.905313 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40207
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:29.976012 9802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:29.976191 9802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:29.976212 9802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:29.977317 9802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:29.977360 9802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:29.977373 9802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:29.977385 9802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:29.977397 9802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:29.978710 9802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:29.978910 9802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:29.979101 9802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:29.983464 9808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:29.983518 9807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:29.983625 9802 server_base.cc:1034] running on GCE node
W20260923 22:08:29.983655 9810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:29.983861 9802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:29.984139 9802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:29.985321 9802 hybrid_clock.cc:648] HybridClock initialized: now 1790201309985308 us; error 26 us; skew 500 ppm
Sep 23 22:08:29 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201309, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:29.987881 9802 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:29.988309 9802 webserver.cc:460] Webserver started at http://127.1.161.190:38559/ using document root <none> and password file <none>
I20260923 22:08:29.988559 9802 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:29.988639 9802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:29.988762 9802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:29.989727 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "c10a505babbf492584416fa787576879"
format_stamp: "Formatted at 2026-09-23 22:08:29 on dist-test-slave-npvd"
server_key: "022c107172bab08a31f8930430a93f6e"
server_key_iv: "d1afeb7f7f8578a9e721b79f3ff5809a"
server_key_version: "encryptionkey@0"
I20260923 22:08:29.990054 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "c10a505babbf492584416fa787576879"
format_stamp: "Formatted at 2026-09-23 22:08:29 on dist-test-slave-npvd"
server_key: "022c107172bab08a31f8930430a93f6e"
server_key_iv: "d1afeb7f7f8578a9e721b79f3ff5809a"
server_key_version: "encryptionkey@0"
I20260923 22:08:29.991291 9802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:29.991931 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:29.992096 9802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:29.992184 9802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "c10a505babbf492584416fa787576879"
format_stamp: "Formatted at 2026-09-23 22:08:29 on dist-test-slave-npvd"
server_key: "022c107172bab08a31f8930430a93f6e"
server_key_iv: "d1afeb7f7f8578a9e721b79f3ff5809a"
server_key_version: "encryptionkey@0"
I20260923 22:08:29.992265 9802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.006443 9802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.008713 9802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.008879 9802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.012420 9802 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40207
I20260923 22:08:30.012449 9869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40207 every 8 connection(s)
I20260923 22:08:30.012753 9802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:30.013550 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:30.015532 9870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Bootstrap starting.
I20260923 22:08:30.016089 9870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:30.016386 9870 log.cc:824] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:30.017038 9870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: No bootstrap required, opened a new log
I20260923 22:08:30.018512 9870 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.018635 9870 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:30.018682 9870 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10a505babbf492584416fa787576879, State: Initialized, Role: FOLLOWER
I20260923 22:08:30.018862 9870 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.018952 9870 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:30.018986 9870 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:30.019040 9870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:30.019061 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9802
I20260923 22:08:30.019199 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:30.019387 1670 external_mini_cluster.cc:1428] Setting key 28063a5b58909aa01bd2b92e1a831544
I20260923 22:08:30.019708 9870 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.019819 9870 leader_election.cc:302] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10a505babbf492584416fa787576879; no voters:
I20260923 22:08:30.019992 9870 leader_election.cc:288] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:30.020169 9874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:30.020231 9870 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:30.020347 9874 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 LEADER]: Becoming Leader. State: Replica: c10a505babbf492584416fa787576879, State: Running, Role: LEADER
I20260923 22:08:30.020442 9874 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.020776 9876 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10a505babbf492584416fa787576879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } } }
I20260923 22:08:30.020859 9876 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:30.021046 9884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:30.021070 9877 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10a505babbf492584416fa787576879. Latest consensus state: current_term: 1 leader_uuid: "c10a505babbf492584416fa787576879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } } }
I20260923 22:08:30.021227 9877 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:30.021569 9884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:30.023108 9884 catalog_manager.cc:1347] Generated new cluster ID: af4db8b036a9421782a850b484f3aa73
I20260923 22:08:30.023154 9884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:30.030371 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.019896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.1.161.190:40207)
0923 22:08:30.020192 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.020199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.020226 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:30.020319 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.020325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.020373 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.021232 (+ 859us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.021335 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.022488 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.023478 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.024080 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.024192 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.026625 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:30.026628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:30.026628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:30.026638 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:30.028995 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.029348 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.029350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.029351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.029434 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.029636 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.029636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.029636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.029866 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:30.029992 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:30.030097 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:08:30.030203 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":93,"threads_started":1}
I20260923 22:08:30.037719 9884 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:30.038344 9884 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:30.044226 9884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Generated new TSK 0
I20260923 22:08:30.044401 9884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:30.079697 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:30.150167 9896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.150329 9896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.150348 9896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.151274 9896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.151304 9896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.151325 9896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:30.152447 9896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.152599 9896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.152740 9896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.157145 9902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.157162 9901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.157181 9896 server_base.cc:1034] running on GCE node
W20260923 22:08:30.157166 9904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.157620 9896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.157847 9896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.158979 9896 hybrid_clock.cc:648] HybridClock initialized: now 1790201310158970 us; error 28 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.161159 9896 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:30.161530 9896 webserver.cc:460] Webserver started at http://127.1.161.129:40537/ using document root <none> and password file <none>
I20260923 22:08:30.161724 9896 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.161783 9896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.161906 9896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:30.162757 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "b0e5bef8d294465ca2ea1ba276f80d4b"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1ddc959513bbc66a5f69161793844992"
server_key_iv: "3e9a7f03a077ac222934a143d9efbd6a"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.163069 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "b0e5bef8d294465ca2ea1ba276f80d4b"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1ddc959513bbc66a5f69161793844992"
server_key_iv: "3e9a7f03a077ac222934a143d9efbd6a"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.164168 9896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.164742 9911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.164872 9896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:30.164953 9896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "b0e5bef8d294465ca2ea1ba276f80d4b"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1ddc959513bbc66a5f69161793844992"
server_key_iv: "3e9a7f03a077ac222934a143d9efbd6a"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.165021 9896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.176577 9896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.178233 9896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.178385 9896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.178584 9896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:30.178859 9896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:30.178912 9896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.178944 9896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:30.178969 9896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.184171 9896 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:35477
I20260923 22:08:30.184213 10024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:35477 every 8 connection(s)
I20260923 22:08:30.184477 9896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:30.191869 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.185058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:58471 (local address 127.1.161.190:40207)
0923 22:08:30.185097 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.185099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.185262 (+ 163us) server_negotiation.cc:407] Connection header received
0923 22:08:30.185613 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.185614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.185624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.185668 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.186412 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.186849 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.187754 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.187941 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.187993 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.189717 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:30.189718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:30.189718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:30.189723 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:30.190986 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.191283 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.191284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.191284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.191327 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.191491 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.191492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.191492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.191567 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:30.191605 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:30.191769 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:08:30.191817 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:30.191995 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.185015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:58471)
0923 22:08:30.185146 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.185178 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.185544 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.185715 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.185717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.186199 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.186346 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.186350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.186931 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.186932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.187680 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.187682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.188023 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.188024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.188133 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.188519 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.188526 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.189642 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:30.191046 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.191050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.191051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.191239 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.191378 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.191378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.191378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.191452 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.191610 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:30.191613 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:30.191695 (+ 82us) client_negotiation.cc:769] Sending connection context
0923 22:08:30.191758 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:08:30.191846 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":79,"threads_started":1}
I20260923 22:08:30.192396 10025 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40207
I20260923 22:08:30.192483 10025 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:30.192639 10025 heartbeater.cc:511] Master 127.1.161.190:40207 requested a full tablet report, sending...
I20260923 22:08:30.193097 9834 ts_manager.cc:194] Registered new tserver with Master: b0e5bef8d294465ca2ea1ba276f80d4b (127.1.161.129:35477)
I20260923 22:08:30.193409 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 9896
I20260923 22:08:30.193508 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:30.193678 1670 external_mini_cluster.cc:1428] Setting key 37f6bfbf3991ec4075433c3db9ae63b8
I20260923 22:08:30.193881 9834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:58471
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:30.217649 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:30.287335 10030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.287506 10030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.287528 10030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.288504 10030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.288533 10030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.288554 10030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:30.289667 10030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.289810 10030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.289942 10030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.294215 10035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.294234 10036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.294231 10038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.294390 10030 server_base.cc:1034] running on GCE node
I20260923 22:08:30.294600 10030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.294839 10030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.295989 10030 hybrid_clock.cc:648] HybridClock initialized: now 1790201310295978 us; error 22 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.298595 10030 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:30.298956 10030 webserver.cc:460] Webserver started at http://127.1.161.130:39727/ using document root <none> and password file <none>
I20260923 22:08:30.299206 10030 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.299280 10030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.299391 10030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:30.300230 10030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "44af9cb0363d43a5b2d45f2498f7ee3e"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1456fc8d0dc5d5eb59df6dba482aef11"
server_key_iv: "6b972aec98c1d8647b4a3c6856bbf724"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.300536 10030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "44af9cb0363d43a5b2d45f2498f7ee3e"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1456fc8d0dc5d5eb59df6dba482aef11"
server_key_iv: "6b972aec98c1d8647b4a3c6856bbf724"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.301641 10030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:08:30.302315 10045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.302593 10030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.302675 10030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "44af9cb0363d43a5b2d45f2498f7ee3e"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1456fc8d0dc5d5eb59df6dba482aef11"
server_key_iv: "6b972aec98c1d8647b4a3c6856bbf724"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.302733 10030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.312557 10030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.314054 10030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.314208 10030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.314407 10030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:30.314685 10030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:30.314738 10030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.314774 10030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:30.314797 10030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.320523 10030 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:43619
I20260923 22:08:30.320567 10158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:43619 every 8 connection(s)
I20260923 22:08:30.320870 10030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:30.321491 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10030
I20260923 22:08:30.321597 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:30.321774 1670 external_mini_cluster.cc:1428] Setting key 3e7cd6a727efffc173f547906200c53b
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
I20260923 22:08:30.330693 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.321527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:35211 (local address 127.1.161.190:40207)
0923 22:08:30.321603 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.321613 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.321811 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:08:30.322389 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.322390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.322399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.322475 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.323379 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.323967 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.325131 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.325456 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.325538 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.328059 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:30.328060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:30.328060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:30.328066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:30.329411 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.329870 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.329871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.329871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.329916 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.330189 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.330190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.330190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.330283 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:30.330331 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:30.330570 (+ 239us) server_negotiation.cc:299] Negotiation successful
0923 22:08:30.330633 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 22:08:30.330888 10161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.321523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:35211)
0923 22:08:30.321681 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.321719 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.322288 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.322544 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.322547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.323051 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.323291 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.323296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.324053 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.324054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.325038 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.325041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.325544 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.325545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.325702 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.326206 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.326212 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.327938 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:30.329495 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.329501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.329503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.329795 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.329984 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.329985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.329986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.330120 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.330350 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:30.330355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:30.330464 (+ 109us) client_negotiation.cc:769] Sending connection context
0923 22:08:30.330542 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 22:08:30.330645 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260923 22:08:30.331452 10159 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40207
I20260923 22:08:30.331528 10159 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:30.331666 10159 heartbeater.cc:511] Master 127.1.161.190:40207 requested a full tablet report, sending...
I20260923 22:08:30.332099 9833 ts_manager.cc:194] Registered new tserver with Master: 44af9cb0363d43a5b2d45f2498f7ee3e (127.1.161.130:43619)
I20260923 22:08:30.332525 9833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:35211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:30.346612 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:30.417901 10164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.418143 10164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.418180 10164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.419243 10164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.419305 10164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.419345 10164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:30.420806 10164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.421031 10164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.421301 10164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.425554 10172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.425559 10169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.425573 10170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.425683 10164 server_base.cc:1034] running on GCE node
I20260923 22:08:30.425948 10164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.426154 10164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.427317 10164 hybrid_clock.cc:648] HybridClock initialized: now 1790201310427312 us; error 31 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.429618 10164 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:30.430022 10164 webserver.cc:460] Webserver started at http://127.1.161.131:40337/ using document root <none> and password file <none>
I20260923 22:08:30.430181 10164 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.430215 10164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.430289 10164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:30.431077 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "b79a376a5ac249a2992e093a59016b03"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "56b17a08b990f3822ff4504d57fa3e79"
server_key_iv: "8468b29d3b52e7ef01a3a0a02305e1fe"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.431401 10164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "b79a376a5ac249a2992e093a59016b03"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "56b17a08b990f3822ff4504d57fa3e79"
server_key_iv: "8468b29d3b52e7ef01a3a0a02305e1fe"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.432435 10164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.432996 10179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.433126 10164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:30.433197 10164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b79a376a5ac249a2992e093a59016b03"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "56b17a08b990f3822ff4504d57fa3e79"
server_key_iv: "8468b29d3b52e7ef01a3a0a02305e1fe"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.433243 10164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.471113 10164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.473133 10164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.473228 10164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.473380 10164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:30.473682 10164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:30.473717 10164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.473737 10164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:30.473776 10164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.479214 10164 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:34745
I20260923 22:08:30.479257 10292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:34745 every 8 connection(s)
I20260923 22:08:30.479530 10164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:30.481117 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10164
I20260923 22:08:30.481230 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:30.481451 1670 external_mini_cluster.cc:1428] Setting key 7c9b502293bad9a805de7a677dd01453
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:30.488036 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.480158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:45463 (local address 127.1.161.190:40207)
0923 22:08:30.480221 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.480222 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.480420 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:08:30.480859 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.480860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.480870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.480916 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.482095 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.482622 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.483552 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.483717 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.483784 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.485705 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:30.485706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:30.485706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:30.485712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:30.486913 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.487324 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.487325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.487325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.487375 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:30.487570 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:30.487570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:30.487571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:30.487675 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:30.487721 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:30.487921 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 22:08:30.487975 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:30.488250 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.480143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:45463)
0923 22:08:30.480318 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.480355 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.480776 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.481033 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.481037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.481719 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.482003 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.482009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.482691 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.482691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.483488 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.483490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.483814 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.483814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.483931 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.484312 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.484319 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.485633 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:30.486994 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.487000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.487001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.487264 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.487433 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:30.487434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:30.487434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:30.487524 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:30.487729 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:30.487733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:30.487834 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 22:08:30.487909 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:08:30.488014 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260923 22:08:30.488677 10293 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40207
I20260923 22:08:30.488771 10293 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:30.488924 10293 heartbeater.cc:511] Master 127.1.161.190:40207 requested a full tablet report, sending...
I20260923 22:08:30.489256 9833 ts_manager.cc:194] Registered new tserver with Master: b79a376a5ac249a2992e093a59016b03 (127.1.161.131:34745)
I20260923 22:08:30.489578 9833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:45463
I20260923 22:08:30.493075 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
Entry for principal oryx/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab.
WARNING: no policy specified for oryx/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
Entry for principal oryx/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab.
WARNING: no policy specified for oryx/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
Entry for principal oryx/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab.
I20260923 22:08:30.558517 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9896
I20260923 22:08:30.562692 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10030
I20260923 22:08:30.566362 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10164
I20260923 22:08:30.570258 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 9802
I20260923 22:08:30.574744 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40207
--webserver_interface=127.1.161.190
--webserver_port=38559
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:30.645998 10302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.646194 10302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.646225 10302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.647429 10302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:30.647470 10302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.647485 10302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.647497 10302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:30.647560 10302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:30.648885 10302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=38559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.649111 10302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.649330 10302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.653697 10307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.653733 10302 server_base.cc:1034] running on GCE node
W20260923 22:08:30.653707 10308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.653707 10310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.654085 10302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.654330 10302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.655486 10302 hybrid_clock.cc:648] HybridClock initialized: now 1790201310655475 us; error 22 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.657819 10302 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:30.658236 10302 webserver.cc:460] Webserver started at http://127.1.161.190:38559/ using document root <none> and password file <none>
I20260923 22:08:30.658483 10302 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.658560 10302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.659789 10302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.660378 10317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.660564 10302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:30.660655 10302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "c10a505babbf492584416fa787576879"
format_stamp: "Formatted at 2026-09-23 22:08:29 on dist-test-slave-npvd"
server_key: "022c107172bab08a31f8930430a93f6e"
server_key_iv: "d1afeb7f7f8578a9e721b79f3ff5809a"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.660956 10302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.677138 10302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.678874 10302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.679010 10302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.682096 10302 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40207
I20260923 22:08:30.682128 10369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40207 every 8 connection(s)
I20260923 22:08:30.682440 10302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:30.683691 10370 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:08:30.684259 10370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Bootstrap starting.
I20260923 22:08:30.688804 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10302
I20260923 22:08:30.689077 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:35477
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:40207
--webserver_port=40537
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:08:30.691985 10370 log.cc:824] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:30.693392 10370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:08:30.693648 10370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Bootstrap complete.
I20260923 22:08:30.695822 10370 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.696058 10370 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10a505babbf492584416fa787576879, State: Initialized, Role: FOLLOWER
I20260923 22:08:30.696280 10370 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.696380 10370 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:30.696431 10370 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:30.696477 10370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:08:30.697430 10370 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.697554 10370 leader_election.cc:302] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10a505babbf492584416fa787576879; no voters:
I20260923 22:08:30.697762 10370 leader_election.cc:288] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:08:30.697794 10374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:08:30.697902 10374 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [term 2 LEADER]: Becoming Leader. State: Replica: c10a505babbf492584416fa787576879, State: Running, Role: LEADER
I20260923 22:08:30.698022 10370 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:30.698050 10374 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } }
I20260923 22:08:30.698419 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c10a505babbf492584416fa787576879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } } }
I20260923 22:08:30.698494 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:30.698773 10376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10a505babbf492584416fa787576879. Latest consensus state: current_term: 2 leader_uuid: "c10a505babbf492584416fa787576879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10a505babbf492584416fa787576879" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40207 } } }
I20260923 22:08:30.698813 10380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:30.698904 10376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:30.699375 10380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:30.699491 10380 catalog_manager.cc:1259] Loaded cluster ID: af4db8b036a9421782a850b484f3aa73
I20260923 22:08:30.699522 10380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:30.700440 10380 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:30.700555 10380 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c10a505babbf492584416fa787576879: Loaded TSK: 0
I20260923 22:08:30.700671 10380 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:08:30.774729 10372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.774892 10372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.774911 10372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.776144 10372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.776193 10372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.776218 10372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:30.777604 10372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.129.keytab
--principal=oryx/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:35477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=40537
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.777833 10372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.778086 10372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.782447 10397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.782454 10399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.782454 10396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.783033 10372 server_base.cc:1034] running on GCE node
I20260923 22:08:30.783253 10372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.783464 10372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.784657 10372 hybrid_clock.cc:648] HybridClock initialized: now 1790201310784642 us; error 28 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.786805 10372 init.cc:377] Logged in from keytab as oryx/127.1.161.129@KRBTEST.COM (short username oryx)
I20260923 22:08:30.787262 10372 webserver.cc:460] Webserver started at http://127.1.161.129:40537/ using document root <none> and password file <none>
I20260923 22:08:30.787495 10372 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.787549 10372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.788807 10372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.789405 10406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.789603 10372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.789693 10372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "b0e5bef8d294465ca2ea1ba276f80d4b"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1ddc959513bbc66a5f69161793844992"
server_key_iv: "3e9a7f03a077ac222934a143d9efbd6a"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.789995 10372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.809702 10372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.811719 10372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.811884 10372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.812125 10372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:30.812398 10372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:30.812453 10372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.812484 10372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:30.812506 10372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.819396 10372 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:35477
I20260923 22:08:30.819435 10519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:35477 every 8 connection(s)
I20260923 22:08:30.819746 10372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:30.823598 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10372
I20260923 22:08:30.823912 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:43619
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:40207
--webserver_port=39727
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.827605 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.820366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:51647)
0923 22:08:30.820511 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.820553 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.820960 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.821371 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.821374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.821854 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.822013 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.822017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.823289 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.823290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.824154 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.824156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.824857 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.824857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.824939 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.825398 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.825405 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.827476 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260923 22:08:30.827822 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.820427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:51647 (local address 127.1.161.190:40207)
0923 22:08:30.820608 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.820615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.820647 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:08:30.821132 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.821138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.821182 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.821327 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.822119 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.823215 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.824238 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.824778 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.824874 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.827634 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:51647: BlockingRecv error: recv got EOF from 127.1.161.129:51647 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
W20260923 22:08:30.827844 10520 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:40207 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.161.190:40207: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.831823 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.828070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:37921)
0923 22:08:30.828110 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.828120 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.828177 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.828327 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.828328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.828436 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.828472 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.828473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.829000 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.829001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.829430 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.829431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.829750 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.829750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.829807 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.830197 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.830202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.831749 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:08:30.831851 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.828101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:37921 (local address 127.1.161.190:40207)
0923 22:08:30.828143 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.828145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.828167 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:08:30.828235 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.828236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.828259 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.828326 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.828541 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.828928 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.829506 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.829673 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.829749 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.831817 (+ 2068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:37921: BlockingRecv error: recv got EOF from 127.1.161.129:37921 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.835832 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.832079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:32897)
0923 22:08:30.832148 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.832159 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.832209 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.832310 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.832311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.832440 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.832471 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.832473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.832967 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.832967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.833640 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.833641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.833902 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.833902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.833947 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.834235 (+ 288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.834239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.835775 (+ 1536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 22:08:30.835916 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.832116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:32897 (local address 127.1.161.190:40207)
0923 22:08:30.832162 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.832163 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.832197 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:08:30.832258 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.832258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.832265 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.832302 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.832531 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.832898 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.833707 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.833844 (+ 137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.833883 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.835846 (+ 1963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:32897: BlockingRecv error: recv got EOF from 127.1.161.129:32897 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 22:08:30.836059 10520 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:08:30.897533 10524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:30.897771 10524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:30.897812 10524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:30.899011 10524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:30.899071 10524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:30.899114 10524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:30.900525 10524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.130.keytab
--principal=oryx/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:43619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=39727
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:30.900764 10524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:30.900970 10524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:30.905664 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.905689 10530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:30.905680 10529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:30.905740 10524 server_base.cc:1034] running on GCE node
I20260923 22:08:30.906013 10524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:30.906215 10524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:30.907351 10524 hybrid_clock.cc:648] HybridClock initialized: now 1790201310907343 us; error 26 us; skew 500 ppm
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201310, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:30.909538 10524 init.cc:377] Logged in from keytab as oryx/127.1.161.130@KRBTEST.COM (short username oryx)
I20260923 22:08:30.909915 10524 webserver.cc:460] Webserver started at http://127.1.161.130:39727/ using document root <none> and password file <none>
I20260923 22:08:30.910104 10524 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:30.910173 10524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:30.911347 10524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:30.911880 10539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.912034 10524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.912115 10524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "44af9cb0363d43a5b2d45f2498f7ee3e"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "1456fc8d0dc5d5eb59df6dba482aef11"
server_key_iv: "6b972aec98c1d8647b4a3c6856bbf724"
server_key_version: "encryptionkey@0"
I20260923 22:08:30.912407 10524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:30.926168 10524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:30.927966 10524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:30.928119 10524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:30.928340 10524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:30.928587 10524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:30.928634 10524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.928669 10524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:30.928691 10524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:30.933897 10524 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:43619
I20260923 22:08:30.933935 10652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:43619 every 8 connection(s)
I20260923 22:08:30.934199 10524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:30.938431 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10524
I20260923 22:08:30.938668 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:34745
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:40207
--webserver_port=40337
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:30.940838 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.934811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:40911 (local address 127.1.161.190:40207)
0923 22:08:30.934883 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.934885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.935037 (+ 152us) server_negotiation.cc:407] Connection header received
0923 22:08:30.935536 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.935537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.935547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.935596 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.936413 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.936851 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.937758 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.937929 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.937976 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.940789 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:40911: BlockingRecv error: recv got EOF from 127.1.161.130:40911 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 22:08:30.940914 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.934784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:40911)
0923 22:08:30.934937 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.934965 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.935455 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.935645 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.935647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.936171 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.936343 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.936347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.936920 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.936921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.937689 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.937691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.938000 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.938001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.938093 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.938599 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.938607 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.940735 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260923 22:08:30.941128 10653 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:40207 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.161.190:40207: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.944917 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.941314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:48975)
0923 22:08:30.941356 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.941365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.941397 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.941494 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.941494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.941615 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.941663 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.941665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.942205 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.942206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.942642 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.942643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.942963 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.942963 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.943017 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.943391 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.943395 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.944882 (+ 1487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:08:30.945053 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.941366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:48975 (local address 127.1.161.190:40207)
0923 22:08:30.941404 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.941405 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.941413 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:30.941442 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.941443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.941450 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.941484 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.941724 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.942133 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.942727 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.942890 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.942962 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.944986 (+ 2024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:48975: BlockingRecv error: recv got EOF from 127.1.161.130:48975 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:30 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:30.949239 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:30.945163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:40905)
0923 22:08:30.945251 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:30.945261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:30.945301 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:30.945408 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:30.945409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:30.945539 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:30.945569 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.945570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.946058 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.946059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.946803 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:30.946805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.947133 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.947134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:30.947217 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.947550 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:30.947556 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:30.949183 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260923 22:08:30.949330 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:30.945247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:40905 (local address 127.1.161.190:40207)
0923 22:08:30.945291 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:30.945292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:30.945300 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:30.945358 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:30.945359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:30.945364 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:30.945402 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:30.945629 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.945990 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.946878 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:30.947058 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:30.947108 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:30.949254 (+ 2146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:40905: BlockingRecv error: recv got EOF from 127.1.161.130:40905 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260923 22:08:30.949447 10653 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:08:31.013288 10656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:31.013461 10656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:31.013482 10656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:31.014690 10656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:31.014729 10656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:31.014755 10656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:31.016287 10656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/krb5kdc/oryx_127.1.161.131.keytab
--principal=oryx/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:34745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=40337
--tserver_master_addrs=127.1.161.190:40207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:31.016520 10656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:31.016743 10656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:31.021425 10656 server_base.cc:1034] running on GCE node
W20260923 22:08:31.021402 10664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:31.021432 10661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:31.021410 10662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:31.021788 10656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:31.022020 10656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:31.023195 10656 hybrid_clock.cc:648] HybridClock initialized: now 1790201311023153 us; error 30 us; skew 500 ppm
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201311, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:31.025564 10656 init.cc:377] Logged in from keytab as oryx/127.1.161.131@KRBTEST.COM (short username oryx)
I20260923 22:08:31.025936 10656 webserver.cc:460] Webserver started at http://127.1.161.131:40337/ using document root <none> and password file <none>
I20260923 22:08:31.026150 10656 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:31.026245 10656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:31.027525 10656 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:31.028082 10671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:31.028270 10656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:31.028357 10656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b79a376a5ac249a2992e093a59016b03"
format_stamp: "Formatted at 2026-09-23 22:08:30 on dist-test-slave-npvd"
server_key: "56b17a08b990f3822ff4504d57fa3e79"
server_key_iv: "8468b29d3b52e7ef01a3a0a02305e1fe"
server_key_version: "encryptionkey@0"
I20260923 22:08:31.028656 10656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:31.038652 10656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:31.040220 10656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:31.040359 10656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:31.040541 10656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:31.040858 10656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:31.040912 10656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:31.040953 10656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:31.040979 10656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:31.046070 10656 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:34745
I20260923 22:08:31.046103 10784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:34745 every 8 connection(s)
I20260923 22:08:31.046411 10656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:31.053052 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.047064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:52803 (local address 127.1.161.190:40207)
0923 22:08:31.047129 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.047131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.047339 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:08:31.047768 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.047769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.047780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.047832 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.048651 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.049270 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.050202 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.050379 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.050440 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.053002 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:52803: BlockingRecv error: recv got EOF from 127.1.161.131:52803 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:31.053102 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.047055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:52803)
0923 22:08:31.047208 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.047248 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.047679 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.047889 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.047892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.048409 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.048561 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.048565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.049377 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.049378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.050122 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.050124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.050476 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.050477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.050580 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.051015 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.051022 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.052943 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260923 22:08:31.053153 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10656
W20260923 22:08:31.053315 10785 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:40207 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.161.190:40207: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.058089 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.053517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:41381)
0923 22:08:31.053572 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.053583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.053621 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.053855 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.053856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.053982 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.054028 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.054029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.054757 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.054757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.055416 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.055418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.055869 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.055870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.055917 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.056311 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.056315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.058047 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 22:08:31.058245 10789 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.053645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:41381 (local address 127.1.161.190:40207)
0923 22:08:31.053740 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.053742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.053750 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:31.053769 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.053770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.053780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.053836 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.054090 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.054684 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.055489 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.055796 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.055865 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.058166 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:41381: BlockingRecv error: recv got EOF from 127.1.161.131:41381 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":45,"threads_started":1}
I20260923 22:08:31.061254 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.053633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.1.161.190:40207)
0923 22:08:31.053667 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.053669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.053797 (+ 128us) server_negotiation.cc:407] Connection header received
0923 22:08:31.053863 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.053864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.053872 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.053925 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.054548 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.055105 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.056118 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.056292 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.056373 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.057972 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:31.057975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:31.057976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:31.057985 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:31.059983 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:31.060324 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:31.060326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:31.060326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:31.060406 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:31.060592 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:31.060593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:31.060593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:31.060854 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:31.060981 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:31.061079 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:08:31.061190 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.063280 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.058550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:55429)
0923 22:08:31.058610 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.058620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.058657 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.059071 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.059072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.059220 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.059253 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.059254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.060038 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.060039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.060764 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.060766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.061189 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.061191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.061252 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.061634 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.061637 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.063216 (+ 1579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 22:08:31.063305 10789 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.058948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:55429 (local address 127.1.161.190:40207)
0923 22:08:31.058996 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.058997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.059003 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:31.059016 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.059016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.059024 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.059060 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.059325 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.059945 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.060845 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.061112 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.061170 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.063277 (+ 2107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:55429: BlockingRecv error: recv got EOF from 127.1.161.131:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 22:08:31.063413 10785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.840564 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.836410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:51479)
0923 22:08:31.836562 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.836572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.836622 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.836785 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.836785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.836925 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.836973 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.836974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.837598 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.837599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.838137 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.838139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.838460 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.838461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.838494 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.838934 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.838938 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.840510 (+ 1572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
W20260923 22:08:31.840623 10791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.836489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:51479 (local address 127.1.161.190:40207)
0923 22:08:31.836611 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.836613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.836621 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:31.836697 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.836697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.836706 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.836768 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.837032 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.837537 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.838208 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.838408 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.838450 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.840576 (+ 2126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:51479: BlockingRecv error: recv got EOF from 127.1.161.129:51479 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:31 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:31.953955 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:31.949791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:36193)
0923 22:08:31.949930 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:31.949941 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:31.949993 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:31.950111 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:31.950111 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:31.950243 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:31.950287 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.950289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.950827 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.950827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.951531 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:31.951534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.951841 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.951842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:31.951875 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.952293 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:31.952298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:31.953895 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":55,"threads_started":1}
W20260923 22:08:31.954069 10791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:31.949951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:36193 (local address 127.1.161.190:40207)
0923 22:08:31.950008 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:31.950009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:31.950028 (+ 19us) server_negotiation.cc:407] Connection header received
0923 22:08:31.950045 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:31.950046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:31.950054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:31.950090 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:31.950338 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.950771 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.951585 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:31.951786 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:31.951835 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:31.953978 (+ 2143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:36193: BlockingRecv error: recv got EOF from 127.1.161.130:36193 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:32.073711 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.063686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:47213)
0923 22:08:32.063797 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.063806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.063851 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.063956 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.063957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.064085 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.064127 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.064128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.064694 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.064694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.065151 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.065153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.065427 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.065427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.065465 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.072091 (+ 6626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:32.072095 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:32.073619 (+ 1524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
W20260923 22:08:32.073740 10791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.063737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:47213 (local address 127.1.161.190:40207)
0923 22:08:32.063774 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.063775 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.063855 (+ 80us) server_negotiation.cc:407] Connection header received
0923 22:08:32.063888 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.063888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.063898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.063944 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.064175 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.064640 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.065207 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.065364 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.065405 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.073684 (+ 8279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:47213: BlockingRecv error: recv got EOF from 127.1.161.131:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:32.845292 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.840960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:44447)
0923 22:08:32.841061 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.841078 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.841125 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.841261 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.841262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.841524 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.841588 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.841592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.842491 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.842492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.842943 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.842945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.843366 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.843367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.843405 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.843782 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:32.843785 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:32.845242 (+ 1457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":58,"threads_started":1}
W20260923 22:08:32.845425 10795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.841003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:44447 (local address 127.1.161.190:40207)
0923 22:08:32.841113 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.841115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.841123 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:32.841174 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.841175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.841186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.841255 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.841683 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.842373 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.843064 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.843296 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.843362 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.845341 (+ 1979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:44447: BlockingRecv error: recv got EOF from 127.1.161.129:44447 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:32 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:32.958909 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:32.954455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:53689)
0923 22:08:32.954635 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:32.954646 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:32.954727 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:32.954856 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:32.954857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:32.955024 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:32.955067 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.955069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.955705 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.955706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.956193 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:32.956194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.956502 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.956503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:32.956553 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.956924 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:32.956928 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:32.958841 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
W20260923 22:08:32.959007 10795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:32.954544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:53689 (local address 127.1.161.190:40207)
0923 22:08:32.954622 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:32.954624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:32.954711 (+ 87us) server_negotiation.cc:407] Connection header received
0923 22:08:32.954777 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:32.954777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:32.954790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:32.954850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:32.955140 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.955610 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.956270 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:32.956440 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:32.956505 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:32.958946 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:53689: BlockingRecv error: recv got EOF from 127.1.161.130:53689 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:33.078861 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.074234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:44389)
0923 22:08:33.074367 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.074378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.074469 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.074605 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.074606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.074772 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.074812 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.074814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.075567 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.075568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.076275 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.076278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.076618 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.076619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.076675 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.077095 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:33.077099 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.078806 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":52,"threads_started":1}
W20260923 22:08:33.078964 10795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:33.074389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:44389 (local address 127.1.161.190:40207)
0923 22:08:33.074443 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.074445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.074453 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:33.074521 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.074522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.074535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.074600 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.074885 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.075472 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.076348 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.076528 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.076581 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.078891 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:44389: BlockingRecv error: recv got EOF from 127.1.161.131:44389 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:33.851019 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.845871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:36283)
0923 22:08:33.845981 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.845991 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.846044 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.846178 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.846179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.846370 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.846433 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.846435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.847431 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.847432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.848139 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.848142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.848734 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.848735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.848793 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.849230 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:33.849235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.850933 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
W20260923 22:08:33.851058 10799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:33.845909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:36283 (local address 127.1.161.190:40207)
0923 22:08:33.846027 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.846029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.846038 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:08:33.846087 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.846088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.846102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.846168 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.846496 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.847358 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.848203 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.848641 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.848718 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.850997 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:36283: BlockingRecv error: recv got EOF from 127.1.161.129:36283 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:33 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:33.964177 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:33.959487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:48981)
0923 22:08:33.959645 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:33.959655 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:33.959734 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:33.959875 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:33.959876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:33.960053 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:33.960098 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.960100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.960903 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.960904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.961390 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:33.961392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.961864 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.961865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:33.961939 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.962364 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:33.962367 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:33.964126 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
W20260923 22:08:33.964315 10799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:33.959565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:48981 (local address 127.1.161.190:40207)
0923 22:08:33.959631 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:33.959632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:33.959715 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:08:33.959796 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:33.959797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:33.959808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:33.959868 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:33.960181 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.960829 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.961474 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:33.961775 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:33.961849 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:33.964202 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:48981: BlockingRecv error: recv got EOF from 127.1.161.130:48981 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:34.083649 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.079407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:40317)
0923 22:08:34.079573 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.079585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.079654 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.079800 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.079801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.079959 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.080009 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.080011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.080588 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.080589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.081051 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.081052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.081442 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.081443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.081497 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.081976 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:34.081980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.083600 (+ 1620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
W20260923 22:08:34.083750 10799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:34.079507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:40317 (local address 127.1.161.190:40207)
0923 22:08:34.079578 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.079580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.079644 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:08:34.079722 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.079723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.079737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.079794 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.080080 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.080508 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.081133 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.081377 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.081449 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.083678 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:40317: BlockingRecv error: recv got EOF from 127.1.161.131:40317 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:34.855114 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.851406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:46935)
0923 22:08:34.851519 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.851529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.851588 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.851708 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.851709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.851849 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.851898 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.851899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.852426 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.852427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.852886 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.852887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.853151 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.853152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.853185 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.853565 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:34.853568 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.855063 (+ 1495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
W20260923 22:08:34.855240 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:34.851459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:46935 (local address 127.1.161.190:40207)
0923 22:08:34.851570 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.851572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.851580 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:34.851636 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.851636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.851646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.851696 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.851951 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.852372 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.852948 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.853100 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.853141 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.855143 (+ 2002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:46935: BlockingRecv error: recv got EOF from 127.1.161.129:46935 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:34 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:34.968792 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:34.964606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:60535)
0923 22:08:34.964719 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:34.964730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:34.964782 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:34.964889 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:34.964889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:34.965027 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:34.965071 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.965072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.965652 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.965652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.966081 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:34.966082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.966349 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.966350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:34.966382 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.966846 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:34.966851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:34.968736 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
W20260923 22:08:34.968884 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:34.964654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:60535 (local address 127.1.161.190:40207)
0923 22:08:34.964711 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:34.964714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:34.964785 (+ 71us) server_negotiation.cc:407] Connection header received
0923 22:08:34.964832 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:34.964832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:34.964844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:34.964880 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:34.965149 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.965590 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.966141 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:34.966297 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:34.966341 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:34.968820 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:60535: BlockingRecv error: recv got EOF from 127.1.161.130:60535 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:35.087946 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.084109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:43675)
0923 22:08:35.084246 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:35.084256 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:35.084307 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:35.084413 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:35.084414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:35.084593 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:35.084656 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.084658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.085322 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.085323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.085764 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.085765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.086024 (+ 259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.086024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.086058 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.086405 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:35.086409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:35.087895 (+ 1486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
W20260923 22:08:35.088047 10803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:35.084155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:43675 (local address 127.1.161.190:40207)
0923 22:08:35.084212 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.084214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.084309 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:08:35.084350 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.084351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.084361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.084396 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.084733 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.085264 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.085824 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.085973 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.086013 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.087981 (+ 1968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:43675: BlockingRecv error: recv got EOF from 127.1.161.131:43675 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:35.859768 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.855554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:47651)
0923 22:08:35.855689 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:35.855699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:35.855749 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:35.855853 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:35.855853 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:35.856003 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:35.856052 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.856054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.856626 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.856627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.857110 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.857111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.857404 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.857405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.857442 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.857982 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:35.857986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:35.859718 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
W20260923 22:08:35.859881 10807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:35.855637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:47651 (local address 127.1.161.190:40207)
0923 22:08:35.855742 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.855744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.855752 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:35.855790 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.855791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.855800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.855847 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.856108 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.856562 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.857180 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.857344 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.857388 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.859786 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:47651: BlockingRecv error: recv got EOF from 127.1.161.129:47651 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:35 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:35.973570 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:35.969315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:51713)
0923 22:08:35.969441 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:35.969450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:35.969513 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:35.969696 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:35.969697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:35.969873 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:35.969930 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.969932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.970568 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.970569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.971058 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:35.971059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.971414 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.971415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:35.971458 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.971851 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:35.971855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:35.973516 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
W20260923 22:08:35.973711 10807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:35.969513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:51713 (local address 127.1.161.190:40207)
0923 22:08:35.969572 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:35.969573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:35.969580 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:35.969599 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:35.969599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:35.969609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:35.969668 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:35.970000 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.970511 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.971124 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:35.971335 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:35.971381 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:35.973621 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:51713: BlockingRecv error: recv got EOF from 127.1.161.130:51713 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:36.092360 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.088391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:46957)
0923 22:08:36.088552 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.088562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.088625 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.088733 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.088734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.088894 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:36.088940 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.088942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.089512 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.089513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.089952 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.089953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.090207 (+ 254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.090208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.090238 (+ 30us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.090604 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:36.090608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:36.092279 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
W20260923 22:08:36.092396 10807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:36.088465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:46957 (local address 127.1.161.190:40207)
0923 22:08:36.088523 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.088525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.088625 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:08:36.088666 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.088667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.088678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.088729 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:36.088994 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.089458 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.090008 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.090155 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.090196 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.092341 (+ 2145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:46957: BlockingRecv error: recv got EOF from 127.1.161.131:46957 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:36.864082 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.860145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:54775)
0923 22:08:36.860241 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.860251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.860304 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.860427 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.860427 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.860562 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:36.860609 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.860611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.861167 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.861168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.861631 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.861632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.861918 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.861918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.861954 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.862395 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:36.862401 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:36.864034 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":47,"threads_started":1}
W20260923 22:08:36.864207 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:36.860208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:54775 (local address 127.1.161.190:40207)
0923 22:08:36.860321 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.860322 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.860340 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:08:36.860357 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.860358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.860366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.860408 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:36.860663 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.861112 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.861691 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.861862 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.861905 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.864137 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:54775: BlockingRecv error: recv got EOF from 127.1.161.129:54775 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:36 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:36.978078 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:36.974040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:45175)
0923 22:08:36.974167 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:36.974177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:36.974239 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:36.974374 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:36.974374 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:36.974522 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:36.974566 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.974568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.975361 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.975362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.975807 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:36.975808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.976098 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.976098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:36.976130 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.976550 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:36.976555 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:36.978028 (+ 1473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
W20260923 22:08:36.978221 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:36.974094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:45175 (local address 127.1.161.190:40207)
0923 22:08:36.974167 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:36.974174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:36.974247 (+ 73us) server_negotiation.cc:407] Connection header received
0923 22:08:36.974290 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:36.974290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:36.974302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:36.974358 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:36.974621 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.975289 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.975869 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:36.976040 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:36.976090 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:36.978132 (+ 2042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:45175: BlockingRecv error: recv got EOF from 127.1.161.130:45175 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:37.101365 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:37.092968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:47769)
0923 22:08:37.093101 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:37.093109 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:37.093157 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:37.093296 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:37.093296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:37.093438 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:37.093480 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.093482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.094159 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.094160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.094709 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.094711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.095297 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.095298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.095362 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.099497 (+ 4135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:37.099502 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:37.101299 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
W20260923 22:08:37.101439 10811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:37.093047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:47769 (local address 127.1.161.190:40207)
0923 22:08:37.093092 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:37.093094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:37.093154 (+ 60us) server_negotiation.cc:407] Connection header received
0923 22:08:37.093226 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:37.093227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:37.093240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:37.093291 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:37.093537 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.094074 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.094831 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.095121 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.095227 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.101380 (+ 6153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:47769: BlockingRecv error: recv got EOF from 127.1.161.131:47769 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:37.868527 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:37.864466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:49505)
0923 22:08:37.864593 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:37.864603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:37.864653 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:37.864768 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:37.864769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:37.864938 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:37.864987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.864988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.865583 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.865584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.866086 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.866088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.866368 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.866369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.866404 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.866809 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:37.866814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:37.868437 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
W20260923 22:08:37.868583 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:37.864551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:49505 (local address 127.1.161.190:40207)
0923 22:08:37.864636 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:37.864638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:37.864645 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:37.864691 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:37.864691 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:37.864701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:37.864750 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:37.865042 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.865521 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.866146 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.866311 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.866354 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.868511 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:49505: BlockingRecv error: recv got EOF from 127.1.161.129:49505 (error 108)
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":41,"threads_started":1}
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:37 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:37.982677 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:37.978666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:52297)
0923 22:08:37.978816 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:37.978828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:37.978908 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:37.979069 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:37.979069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:37.979296 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:37.979368 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.979371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.979960 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.979960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.980480 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:37.980481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.980767 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.980768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:37.980804 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.981163 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:37.981166 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:37.982625 (+ 1459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
W20260923 22:08:37.982761 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:37.978716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:52297 (local address 127.1.161.190:40207)
0923 22:08:37.978789 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:37.978792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:37.978910 (+ 118us) server_negotiation.cc:407] Connection header received
0923 22:08:37.978983 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:37.978984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:37.978997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:37.979055 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:37.979437 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.979899 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.980544 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:37.980710 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:37.980751 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:37.982705 (+ 1954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:52297: BlockingRecv error: recv got EOF from 127.1.161.130:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:38.105679 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:38.101842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:36947)
0923 22:08:38.101943 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:38.101953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:38.102018 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:38.102124 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:38.102125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:38.102265 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:38.102314 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.102322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.102842 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.102842 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.103389 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.103391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.103659 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.103659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.103692 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.104066 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:38.104069 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:38.105624 (+ 1555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":59,"threads_started":1}
W20260923 22:08:38.105744 10815 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:38.101875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:36947 (local address 127.1.161.190:40207)
0923 22:08:38.101929 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:38.101931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:38.102009 (+ 78us) server_negotiation.cc:407] Connection header received
0923 22:08:38.102064 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:38.102064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:38.102075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:38.102120 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:38.102369 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.102775 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.103456 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.103605 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.103643 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.105702 (+ 2059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:36947: BlockingRecv error: recv got EOF from 127.1.161.131:36947 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:38.872907 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:38.868962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:40671)
0923 22:08:38.869096 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:38.869106 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:38.869154 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:38.869304 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:38.869305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:38.869459 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:38.869506 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.869508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.870085 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.870086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.870531 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.870532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.870800 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.870801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.870833 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.871246 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:38.871250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:38.872811 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260923 22:08:38.872946 10819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:38.869078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:40671 (local address 127.1.161.190:40207)
0923 22:08:38.869191 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:38.869193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:38.869200 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:38.869220 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:38.869220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:38.869229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:38.869285 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:38.869567 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.870029 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.870586 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.870742 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.870783 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.872880 (+ 2097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:40671: BlockingRecv error: recv got EOF from 127.1.161.129:40671 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:38 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:38.987828 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:38.983208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:48515)
0923 22:08:38.983374 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:38.983384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:38.983474 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:38.983608 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:38.983609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:38.983762 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:38.983803 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.983805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.984577 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.984579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.985096 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:38.985098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.985483 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.985484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:38.985547 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.986095 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:38.986099 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:38.987751 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
W20260923 22:08:38.987943 10819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:38.983273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:48515 (local address 127.1.161.190:40207)
0923 22:08:38.983361 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:38.983363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:38.983475 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:08:38.983528 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:38.983528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:38.983540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:38.983604 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:38.983873 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.984496 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.985184 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:38.985381 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:38.985473 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:38.987847 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:48515: BlockingRecv error: recv got EOF from 127.1.161.130:48515 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:39.110577 10821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:39.106108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:52499)
0923 22:08:39.106254 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:39.106264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:39.106329 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:39.106463 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:39.106464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:39.106627 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:39.106670 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.106672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.107316 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.107318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.107817 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.107819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.108170 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.108171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.108224 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.108600 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:39.108604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:39.110475 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
W20260923 22:08:39.110603 10819 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:39.106171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:52499 (local address 127.1.161.190:40207)
0923 22:08:39.106239 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:39.106241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:39.106309 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:08:39.106387 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:39.106388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:39.106402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:39.106457 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:39.106747 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.107253 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.107920 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.108090 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.108169 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.110544 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:52499: BlockingRecv error: recv got EOF from 127.1.161.131:52499 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:39.877338 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:39.873361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:47225)
0923 22:08:39.873484 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:39.873493 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:39.873561 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:39.873681 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:39.873682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:39.873820 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:39.873866 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.873867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.874462 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.874463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.874905 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.874907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.875228 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.875228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.875283 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.875719 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:39.875724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:39.877288 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260923 22:08:39.877490 10823 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:39.873406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:47225 (local address 127.1.161.190:40207)
0923 22:08:39.873514 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:39.873516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:39.873543 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:08:39.873608 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:39.873608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:39.873619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:39.873669 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:39.873927 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.874401 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.874976 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.875159 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.875225 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.877402 (+ 2177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:47225: BlockingRecv error: recv got EOF from 127.1.161.129:47225 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":58,"threads_started":1}
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:39 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 22:08:39.992827 10823 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:39.988365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:43221 (local address 127.1.161.190:40207)
0923 22:08:39.988411 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:39.988414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:39.988542 (+ 128us) server_negotiation.cc:407] Connection header received
0923 22:08:39.988597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:39.988598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:39.988608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:39.988648 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:39.988947 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.989426 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.990089 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.990265 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.990313 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.992731 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:43221: BlockingRecv error: recv got EOF from 127.1.161.130:43221 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:08:39.992880 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:39.988286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:43221)
0923 22:08:39.988451 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:39.988464 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:39.988534 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:39.988675 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:39.988676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:39.988836 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:39.988885 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.988886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.989507 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.989508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.990019 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:39.990020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:39.990327 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:39.990327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:39.990364 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:39.990787 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:39.990791 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:39.992644 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:40.114868 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:40.111026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:39105)
0923 22:08:40.111191 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:40.111200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:40.111258 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:40.111369 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:40.111369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:40.111497 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:40.111539 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.111540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.112067 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.112067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.112497 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.112498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.112763 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.112764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.112795 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.113217 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:40.113221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:40.114819 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
W20260923 22:08:40.115020 10823 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:40.111080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:39105 (local address 127.1.161.190:40207)
0923 22:08:40.111136 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:40.111138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:40.111255 (+ 117us) server_negotiation.cc:407] Connection header received
0923 22:08:40.111295 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:40.111295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:40.111305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:40.111354 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:40.111593 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.112011 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.112557 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.112710 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.112755 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.114894 (+ 2139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:39105: BlockingRecv error: recv got EOF from 127.1.161.131:39105 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:40.881868 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:40.877729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:59681)
0923 22:08:40.877860 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:40.877869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:40.877911 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:40.878054 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:40.878055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:40.878204 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:40.878248 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.878250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.878797 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.878797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.879373 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.879375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.879698 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.879698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.879740 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.880237 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:40.880241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:40.881785 (+ 1544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260923 22:08:40.881910 10827 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:40.877786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:59681 (local address 127.1.161.190:40207)
0923 22:08:40.877907 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:40.877909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:40.877939 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:08:40.877958 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:40.877958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:40.877984 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:40.878029 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:40.878307 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.878740 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.879444 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.879636 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.879688 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.881850 (+ 2162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:59681: BlockingRecv error: recv got EOF from 127.1.161.129:59681 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:40 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:40.997231 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:40.993272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:40213)
0923 22:08:40.993412 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:40.993427 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:40.993480 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:40.993585 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:40.993586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:40.993726 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:40.993768 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.993770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.994312 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.994312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.994774 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:40.994775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.995077 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.995077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:40.995109 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.995572 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:40.995577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:40.997143 (+ 1566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
W20260923 22:08:40.997262 10827 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:40.993376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:40213 (local address 127.1.161.190:40207)
0923 22:08:40.993434 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:40.993436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:40.993467 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:08:40.993516 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:40.993517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:40.993527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:40.993567 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:40.993830 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.994254 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.994842 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:40.995009 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:40.995068 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:40.997209 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:40213: BlockingRecv error: recv got EOF from 127.1.161.130:40213 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:41.119146 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:41.115290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:37937)
0923 22:08:41.115400 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:41.115410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:41.115459 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:41.115579 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:41.115580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:41.115769 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:41.115812 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.115814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.116436 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.116436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.116948 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.116949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.117250 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.117251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.117284 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:41.117702 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:41.117706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:41.119085 (+ 1379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
W20260923 22:08:41.119292 10827 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:41.115369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:37937 (local address 127.1.161.190:40207)
0923 22:08:41.115434 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:41.115436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:41.115450 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:08:41.115507 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:41.115508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:41.115517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:41.115562 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:41.115867 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.116372 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.117010 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.117197 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.117238 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:41.119221 (+ 1983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:37937: BlockingRecv error: recv got EOF from 127.1.161.131:37937 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:41.886591 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:41.882402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:41865)
0923 22:08:41.882556 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:41.882569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:41.882650 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:41.882773 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:41.882773 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:41.882924 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:41.882970 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.882971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.883572 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.883573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.884087 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.884089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.884366 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.884366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.884405 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:41.884789 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:41.884793 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:41.886517 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
W20260923 22:08:41.886698 10831 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:41.882471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:41865 (local address 127.1.161.190:40207)
0923 22:08:41.882609 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:41.882610 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:41.882621 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:08:41.882692 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:41.882692 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:41.882701 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:41.882752 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:41.883029 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.883512 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.884151 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.884309 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.884351 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:41.886607 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:41865: BlockingRecv error: recv got EOF from 127.1.161.129:41865 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:41 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:42.001855 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:41.997650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:46583)
0923 22:08:41.997865 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:41.997875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:41.997934 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:41.998054 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:41.998054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:41.998206 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:41.998254 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.998255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.998846 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.998847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.999473 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:41.999475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.999811 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.999812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:41.999872 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.000262 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:42.000267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:42.001788 (+ 1521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
W20260923 22:08:42.001961 10831 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:41.997699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:46583 (local address 127.1.161.190:40207)
0923 22:08:41.997740 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:41.997742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:41.997938 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:08:41.997983 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:41.997983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:41.997994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:41.998035 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:41.998309 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.998773 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.999544 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:41.999723 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:41.999800 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.001891 (+ 2091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:46583: BlockingRecv error: recv got EOF from 127.1.161.130:46583 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 23 22:08:42 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:42 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:42.126852 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:42.119650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:40749)
0923 22:08:42.119811 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:42.119823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:42.119887 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:42.120026 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:42.120027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:42.120199 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:42.120238 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:42.120240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.120835 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.120836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:42.121301 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:42.121303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.121619 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.121619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:42.121670 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.125226 (+ 3556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:42.125231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:42.126791 (+ 1560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
W20260923 22:08:42.126948 10831 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:42.119722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:40749 (local address 127.1.161.190:40207)
0923 22:08:42.119798 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:42.119800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:42.119869 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:08:42.119938 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:42.119939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:42.119951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:42.120022 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:42.120319 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.120764 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.121379 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.121545 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.121617 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.126882 (+ 5265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:40749: BlockingRecv error: recv got EOF from 127.1.161.131:40749 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 22:08:42 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:42 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:42.891561 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:42.887095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:59543)
0923 22:08:42.887261 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:42.887270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:42.887319 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:42.887483 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:42.887484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:42.887644 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:42.887686 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:42.887687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.888321 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.888322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:42.888823 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:42.888824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.889171 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.889171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:42.889208 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.889652 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:42.889657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:42.891509 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
W20260923 22:08:42.891774 10835 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:42.887233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:59543 (local address 127.1.161.190:40207)
0923 22:08:42.887360 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:42.887362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:42.887377 (+ 15us) server_negotiation.cc:407] Connection header received
0923 22:08:42.887397 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:42.887397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:42.887407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:42.887470 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:42.887759 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.888222 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.888905 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:42.889102 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:42.889165 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:42.891692 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:59543: BlockingRecv error: recv got EOF from 127.1.161.129:59543 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:43.005968 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:43.002253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:50965)
0923 22:08:43.002381 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:43.002389 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:43.002435 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:43.002539 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:43.002540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:43.002666 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:43.002725 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.002726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.003277 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.003278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.003742 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.003744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.004073 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.004074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.004107 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.004458 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:43.004461 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:43.005920 (+ 1459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
W20260923 22:08:43.006078 10835 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:43.002295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:50965 (local address 127.1.161.190:40207)
0923 22:08:43.002352 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:43.002353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:43.002436 (+ 83us) server_negotiation.cc:407] Connection header received
0923 22:08:43.002480 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:43.002480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:43.002489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:43.002522 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:43.002781 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.003222 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.003825 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.004010 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.004065 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.006009 (+ 1944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:50965: BlockingRecv error: recv got EOF from 127.1.161.130:50965 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:43.131433 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:43.127301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:49725)
0923 22:08:43.127426 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:43.127435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:43.127495 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:43.127604 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:43.127605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:43.127764 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:43.127801 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.127802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.128376 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.128377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.128961 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.128963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.129288 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.129289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.129372 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.129777 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:43.129781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:43.131354 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260923 22:08:43.131482 10835 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:43.127372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:49725 (local address 127.1.161.190:40207)
0923 22:08:43.127413 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:43.127415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:43.127478 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:08:43.127546 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:43.127547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:43.127558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:43.127601 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:43.127870 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.128294 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.129030 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.129215 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.129287 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.131429 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:49725: BlockingRecv error: recv got EOF from 127.1.161.131:49725 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:43 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:43.900600 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:43.892038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:34431)
0923 22:08:43.892176 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:43.892188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:43.892249 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:43.895373 (+ 3124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:43.895375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:43.895569 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:43.895622 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.895624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.896428 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.896429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.897247 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:43.897250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.897713 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.897714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:43.897769 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.898270 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:43.898276 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:43.900498 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
W20260923 22:08:43.900641 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:43.892091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:34431 (local address 127.1.161.190:40207)
0923 22:08:43.895232 (+ 3141us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:43.895233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:43.895243 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:43.895283 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:43.895283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:43.895294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:43.895355 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:43.895689 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.896374 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.897345 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:43.897652 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:43.897715 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:43.900569 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:34431: BlockingRecv error: recv got EOF from 127.1.161.129:34431 (error 108)
Metrics: {"server-negotiator.queue_time_us":3121,"thread_start_us":61,"threads_started":1}
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:44.010619 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:44.006398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:49655)
0923 22:08:44.006620 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:44.006631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:44.006685 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:44.006825 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:44.006826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:44.006968 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:44.007013 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.007014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.007584 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.007584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.008027 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.008029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.008342 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.008342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.008404 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.008815 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:44.008818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:44.010567 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":58,"threads_started":1}
W20260923 22:08:44.010768 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:44.006582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:49655 (local address 127.1.161.190:40207)
0923 22:08:44.006673 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:44.006675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:44.006685 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:08:44.006733 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:44.006734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:44.006744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:44.006813 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:44.007072 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.007523 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.008100 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.008270 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.008427 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.010672 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:49655: BlockingRecv error: recv got EOF from 127.1.161.130:49655 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:44.135529 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:44.131822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:39175)
0923 22:08:44.131966 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:44.131974 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:44.132024 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:44.132129 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:44.132129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:44.132256 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:44.132294 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.132296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.132854 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.132854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.133302 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.133303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.133578 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.133578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.133612 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.133995 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:44.133998 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:44.135474 (+ 1476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260923 22:08:44.135668 10839 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:44.131870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:39175 (local address 127.1.161.190:40207)
0923 22:08:44.131931 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:44.131933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:44.132026 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:08:44.132061 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:44.132062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:44.132073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:44.132119 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:44.132353 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.132777 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.133372 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.133524 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.133571 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.135594 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:39175: BlockingRecv error: recv got EOF from 127.1.161.131:39175 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:44 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:44.904891 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:44.901006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:40411)
0923 22:08:44.901125 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:44.901133 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:44.901175 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:44.901323 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:44.901323 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:44.901473 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:44.901518 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.901519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.902071 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.902071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.902513 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:44.902515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.902795 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.902796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:44.902829 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.903235 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:44.903238 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:44.904838 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
W20260923 22:08:44.905019 10843 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:44.901070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:40411 (local address 127.1.161.190:40207)
0923 22:08:44.901213 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:44.901214 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:44.901222 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:44.901241 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:44.901241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:44.901252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:44.901310 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:44.901578 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.902012 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.902578 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:44.902740 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:44.902787 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:44.904917 (+ 2130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:40411: BlockingRecv error: recv got EOF from 127.1.161.129:40411 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:45.015887 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:45.011275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:59069)
0923 22:08:45.011439 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:45.011453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:45.011523 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:45.011634 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:45.011634 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:45.011820 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:45.011867 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.011869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.012468 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.012469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.012967 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.012968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.013312 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.013313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.013369 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.013799 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:45.013803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:45.015788 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
W20260923 22:08:45.015938 10843 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:45.011338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:59069 (local address 127.1.161.190:40207)
0923 22:08:45.011412 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:45.011415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:45.011504 (+ 89us) server_negotiation.cc:407] Connection header received
0923 22:08:45.011576 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:45.011577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:45.011588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:45.011631 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:45.011946 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.012393 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.013042 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.013236 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.013304 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.015861 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:59069: BlockingRecv error: recv got EOF from 127.1.161.130:59069 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.131@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:45.139972 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:45.135886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.131:34299)
0923 22:08:45.136011 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:45.136020 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:45.136071 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:45.136184 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:45.136185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:45.136328 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:45.136370 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.136371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.136956 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.136956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.137477 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.137479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.137771 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.137772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.137820 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.138280 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:45.138284 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:45.139910 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
W20260923 22:08:45.140080 10843 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:45.135961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:34299 (local address 127.1.161.190:40207)
0923 22:08:45.136019 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:45.136021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:45.136068 (+ 47us) server_negotiation.cc:407] Connection header received
0923 22:08:45.136111 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:45.136112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:45.136121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:45.136170 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:45.136431 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.136899 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.137542 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.137713 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.137759 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.139988 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.131:34299: BlockingRecv error: recv got EOF from 127.1.161.131:34299 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:45 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.129@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:45.909099 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:45.905327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.129:51331)
0923 22:08:45.905463 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:45.905475 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:45.905538 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:45.905662 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:45.905663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:45.905809 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:45.905854 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.905856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.906413 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.906414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.906900 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:45.906902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.907207 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.907208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:45.907256 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.907600 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:45.907604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:45.909050 (+ 1446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
W20260923 22:08:45.909195 10847 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:45.905379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:51331 (local address 127.1.161.190:40207)
0923 22:08:45.905518 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:45.905520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:45.905527 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:45.905581 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:45.905582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:45.905592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:45.905647 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:45.905911 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.906349 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.906963 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:45.907121 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:45.907192 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:45.909126 (+ 1934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.129:51331: BlockingRecv error: recv got EOF from 127.1.161.129:51331 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
Sep 23 22:08:46 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 22:08:46 dist-test-slave-npvd krb5kdc[9789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.161.130@KRBTEST.COM for oryx/127.1.161.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 22:08:46.020857 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.016369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40207 (local address 127.1.161.130:41421)
0923 22:08:46.016526 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.016543 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.016602 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.016804 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.016805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.016961 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.017018 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.017019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.017628 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.017629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.018152 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.018153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.018442 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.018443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.018481 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.018877 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:46.018881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.020803 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.161.190:40207: Server oryx/127.1.161.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
W20260923 22:08:46.020991 10847 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:46.016443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:41421 (local address 127.1.161.190:40207)
0923 22:08:46.016542 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.016544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.016587 (+ 43us) server_negotiation.cc:407] Connection header received
0923 22:08:46.016666 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.016666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.016676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.016749 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.017085 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.017566 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.018216 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.018384 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.018430 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.020912 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.161.130:41421: BlockingRecv error: recv got EOF from 127.1.161.130:41421 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 22:08:46.062520 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10372
I20260923 22:08:46.066592 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10524
I20260923 22:08:46.070077 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10656
I20260923 22:08:46.073634 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10302
2026-09-23T22:08:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17198 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T22:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:46Z Disabled control of system clock
I20260923 22:08:46.098807 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36659
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:42587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 22:08:46.168872 10852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.169031 10852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.169049 10852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.170033 10852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:46.170066 10852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.170078 10852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.170091 10852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:46.170104 10852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:46.171391 10852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.171545 10852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.171730 10852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.176154 10858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.176178 10860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.176201 10857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.176254 10852 server_base.cc:1034] running on GCE node
I20260923 22:08:46.176563 10852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.176816 10852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.177954 10852 hybrid_clock.cc:648] HybridClock initialized: now 1790201326177944 us; error 22 us; skew 500 ppm
I20260923 22:08:46.179001 10852 webserver.cc:460] Webserver started at http://127.1.161.190:43317/ using document root <none> and password file <none>
I20260923 22:08:46.179283 10852 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.179329 10852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.179476 10852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.180441 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "907314f223b7455abde8026d7fd1c9d6"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "f4681d1733364c4386920fa5d9e8cc63"
server_key_iv: "2e1c0f62bdbb4d68e4efa046cd5d9153"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.180819 10852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "907314f223b7455abde8026d7fd1c9d6"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "f4681d1733364c4386920fa5d9e8cc63"
server_key_iv: "2e1c0f62bdbb4d68e4efa046cd5d9153"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.182080 10852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:46.182669 10866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.182843 10852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:46.182914 10852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "907314f223b7455abde8026d7fd1c9d6"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "f4681d1733364c4386920fa5d9e8cc63"
server_key_iv: "2e1c0f62bdbb4d68e4efa046cd5d9153"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.182956 10852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.192266 10852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.192441 10852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.192519 10852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.195607 10852 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36659
I20260923 22:08:46.195632 10918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36659 every 8 connection(s)
I20260923 22:08:46.195883 10852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:46.196496 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:46.198529 10919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6: Bootstrap starting.
I20260923 22:08:46.199018 10919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:46.199314 10919 log.cc:824] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:46.199855 10919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6: No bootstrap required, opened a new log
I20260923 22:08:46.201318 10919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } }
I20260923 22:08:46.201447 10919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:46.201488 10919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907314f223b7455abde8026d7fd1c9d6, State: Initialized, Role: FOLLOWER
I20260923 22:08:46.201635 10919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } }
I20260923 22:08:46.201740 10919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:46.201777 10919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:46.201818 10919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:46.202217 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10852
I20260923 22:08:46.202282 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:46.202251 10919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } }
I20260923 22:08:46.202363 10919 leader_election.cc:302] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907314f223b7455abde8026d7fd1c9d6; no voters:
I20260923 22:08:46.202467 1670 external_mini_cluster.cc:1428] Setting key de42373d191c6669acb8258ff3c2e649
I20260923 22:08:46.202524 10919 leader_election.cc:288] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:46.202601 10922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:46.202749 10919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:46.202809 10922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [term 1 LEADER]: Becoming Leader. State: Replica: 907314f223b7455abde8026d7fd1c9d6, State: Running, Role: LEADER
I20260923 22:08:46.202931 10922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } }
I20260923 22:08:46.203409 10923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "907314f223b7455abde8026d7fd1c9d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } } }
I20260923 22:08:46.203490 10923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:46.203665 10924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907314f223b7455abde8026d7fd1c9d6. Latest consensus state: current_term: 1 leader_uuid: "907314f223b7455abde8026d7fd1c9d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907314f223b7455abde8026d7fd1c9d6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36659 } } }
I20260923 22:08:46.203737 10933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:46.203809 10924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:46.204315 10933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:46.205477 10933 catalog_manager.cc:1347] Generated new cluster ID: ce99fd8c19ae4cb4a272568d21a884b3
I20260923 22:08:46.205519 10933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:46.208405 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.203023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.1.161.190:36659)
0923 22:08:46.203240 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.203850 (+ 610us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.203980 (+ 130us) server_negotiation.cc:407] Connection header received
0923 22:08:46.204085 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.204091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.204174 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.204512 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.204598 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.205709 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.206620 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.207148 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.207304 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.207837 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.207841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.207841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.208007 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.208042 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.208054 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.208114 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.208169 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.208236 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260923 22:08:46.223206 10933 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:46.223793 10933 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:46.230731 10933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907314f223b7455abde8026d7fd1c9d6: Generated new TSK 0
I20260923 22:08:46.230875 10933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:46.235705 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--builtin_ntp_servers=127.1.161.148:42587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:08:46.306834 10943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.307019 10943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.307049 10943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.308075 10943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.308130 10943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.308166 10943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:46.309366 10943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.309580 10943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.309821 10943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.314618 10948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.314599 10951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.314623 10943 server_base.cc:1034] running on GCE node
W20260923 22:08:46.314608 10949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.314953 10943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.315222 10943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.316385 10943 hybrid_clock.cc:648] HybridClock initialized: now 1790201326316367 us; error 26 us; skew 500 ppm
I20260923 22:08:46.317423 10943 webserver.cc:460] Webserver started at http://127.1.161.129:38127/ using document root <none> and password file <none>
I20260923 22:08:46.317590 10943 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.317625 10943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.317706 10943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.318539 10943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "6e1a1e6c4c6c44b79edafa4db7f43c55"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7eae9ea0526d928ec4a90bb430b3465a"
server_key_iv: "b14c8800858dfb6bdb8af93ddd9253a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.318794 10943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "6e1a1e6c4c6c44b79edafa4db7f43c55"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7eae9ea0526d928ec4a90bb430b3465a"
server_key_iv: "b14c8800858dfb6bdb8af93ddd9253a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.319978 10943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:46.320640 10957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.320849 10943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.320916 10943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "6e1a1e6c4c6c44b79edafa4db7f43c55"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7eae9ea0526d928ec4a90bb430b3465a"
server_key_iv: "b14c8800858dfb6bdb8af93ddd9253a5"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.320998 10943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.331749 10943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.331982 10943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.332110 10943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.332285 10943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:46.332567 10943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:46.332616 10943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.332652 10943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:46.332679 10943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.338061 10943 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37657
I20260923 22:08:46.338124 11070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37657 every 8 connection(s)
I20260923 22:08:46.338425 10943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:46.339547 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 10943
I20260923 22:08:46.339632 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:46.339833 1670 external_mini_cluster.cc:1428] Setting key 5484b48a7847b8a4ee83219e1a996c70
I20260923 22:08:46.340843 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--builtin_ntp_servers=127.1.161.148:42587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 22:08:46.343528 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.339119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:60337 (local address 127.1.161.190:36659)
0923 22:08:46.339211 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.339213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.339386 (+ 173us) server_negotiation.cc:407] Connection header received
0923 22:08:46.339910 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.339911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.339925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.340004 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.340745 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.341316 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.342183 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.342378 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.342456 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.343284 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.343285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.343286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.343290 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.343304 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.343308 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.343357 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.343429 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.343457 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:08:46.343746 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.339076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36659 (local address 127.1.161.129:60337)
0923 22:08:46.339283 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.339312 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.339794 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.340080 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.340083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.340099 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.340653 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.340658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.341396 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.341397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.342097 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.342099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.342464 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.342466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.342590 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.343096 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:46.343097 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.343120 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:46.343138 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:46.343361 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:46.343365 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:46.343368 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:46.343430 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:08:46.343516 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260923 22:08:46.344190 11071 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36659
I20260923 22:08:46.344274 11071 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:46.344424 11071 heartbeater.cc:511] Master 127.1.161.190:36659 requested a full tablet report, sending...
I20260923 22:08:46.344868 10883 ts_manager.cc:194] Registered new tserver with Master: 6e1a1e6c4c6c44b79edafa4db7f43c55 (127.1.161.129:37657)
I20260923 22:08:46.345402 10883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.129:60337
W20260923 22:08:46.412582 11074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.412796 11074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.412818 11074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.413800 11074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.413839 11074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.413862 11074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:46.415086 11074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.415319 11074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.415542 11074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.419757 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.419752 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.419780 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.419816 11074 server_base.cc:1034] running on GCE node
I20260923 22:08:46.420161 11074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.420349 11074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.421490 11074 hybrid_clock.cc:648] HybridClock initialized: now 1790201326421471 us; error 33 us; skew 500 ppm
I20260923 22:08:46.422585 11074 webserver.cc:460] Webserver started at http://127.1.161.130:45299/ using document root <none> and password file <none>
I20260923 22:08:46.422816 11074 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.422871 11074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.422991 11074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.423929 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "638e7092f68e4138881999f4e45fc9b0"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "d7dedd329da65b822dfa50b62ccfd52e"
server_key_iv: "97e69e530844fe4eb2c67acfceacce9f"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.424258 11074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "638e7092f68e4138881999f4e45fc9b0"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "d7dedd329da65b822dfa50b62ccfd52e"
server_key_iv: "97e69e530844fe4eb2c67acfceacce9f"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.425681 11074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:46.426298 11088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.426448 11074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.426541 11074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "638e7092f68e4138881999f4e45fc9b0"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "d7dedd329da65b822dfa50b62ccfd52e"
server_key_iv: "97e69e530844fe4eb2c67acfceacce9f"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.426615 11074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.453429 11074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.453656 11074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.453785 11074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.453965 11074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:46.454246 11074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:46.454298 11074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.454331 11074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:46.454356 11074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.459785 11074 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:44139
I20260923 22:08:46.459815 11201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:44139 every 8 connection(s)
I20260923 22:08:46.460134 11074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:46.464272 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.460764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:34157 (local address 127.1.161.190:36659)
0923 22:08:46.460812 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.460814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.461087 (+ 273us) server_negotiation.cc:407] Connection header received
0923 22:08:46.461535 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.461536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.461545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.461593 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.462096 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.462494 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.463339 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.463491 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.463535 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.464106 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.464107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.464107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.464111 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.464121 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.464124 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.464158 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.464205 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.464223 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:08:46.464465 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.460755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36659 (local address 127.1.161.130:34157)
0923 22:08:46.460926 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.460977 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.461449 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.461635 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.461638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.461645 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.462037 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.462040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.462559 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.462560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.463272 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.463274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.463552 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.463553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.463641 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.464000 (+ 359us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:46.464001 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.464023 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:46.464040 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:46.464160 (+ 120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:46.464162 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:46.464164 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:46.464207 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:08:46.464266 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260923 22:08:46.464934 11202 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36659
I20260923 22:08:46.465015 11202 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:46.465174 11202 heartbeater.cc:511] Master 127.1.161.190:36659 requested a full tablet report, sending...
I20260923 22:08:46.465257 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11074
I20260923 22:08:46.465360 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:46.465468 10883 ts_manager.cc:194] Registered new tserver with Master: 638e7092f68e4138881999f4e45fc9b0 (127.1.161.130:44139)
I20260923 22:08:46.465541 1670 external_mini_cluster.cc:1428] Setting key fdf4f718b78c71a807d07a9c06e5ff04
I20260923 22:08:46.465844 10883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.130:34157
I20260923 22:08:46.466570 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--builtin_ntp_servers=127.1.161.148:42587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 22:08:46.538556 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.538709 11205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.538726 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.539633 11205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.539660 11205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.539680 11205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:46.541113 11205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.541296 11205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.541488 11205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.545881 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.545882 11211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.545881 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.546164 11205 server_base.cc:1034] running on GCE node
I20260923 22:08:46.546331 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.546495 11205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.547647 11205 hybrid_clock.cc:648] HybridClock initialized: now 1790201326547636 us; error 22 us; skew 500 ppm
I20260923 22:08:46.548687 11205 webserver.cc:460] Webserver started at http://127.1.161.131:37423/ using document root <none> and password file <none>
I20260923 22:08:46.548895 11205 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.548956 11205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.549104 11205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.550012 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "f6416b15d11a47ccb2c8228f1c563ec4"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "37ea94862285dca776da92e9a85e162c"
server_key_iv: "7161d417ff5c83b1de45ccd75b695384"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.550369 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "f6416b15d11a47ccb2c8228f1c563ec4"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "37ea94862285dca776da92e9a85e162c"
server_key_iv: "7161d417ff5c83b1de45ccd75b695384"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.551717 11205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:46.552333 11219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.552469 11205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.552567 11205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "f6416b15d11a47ccb2c8228f1c563ec4"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "37ea94862285dca776da92e9a85e162c"
server_key_iv: "7161d417ff5c83b1de45ccd75b695384"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.552630 11205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.568840 11205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.569073 11205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.569211 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.569398 11205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:46.569664 11205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:46.569713 11205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.569751 11205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:46.569775 11205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.575757 11205 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:46681
I20260923 22:08:46.575785 11332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:46681 every 8 connection(s)
I20260923 22:08:46.576061 11205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:46.580268 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.576715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:41761 (local address 127.1.161.190:36659)
0923 22:08:46.576760 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.576762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.576980 (+ 218us) server_negotiation.cc:407] Connection header received
0923 22:08:46.577378 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.577379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.577387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.577433 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.577957 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.578388 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.579253 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.579403 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.579442 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.580074 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.580075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.580075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.580079 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.580090 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.580094 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.580133 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.580194 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.580213 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:08:46.580485 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.576681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36659 (local address 127.1.161.131:41761)
0923 22:08:46.576873 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.576906 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.577301 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.577478 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.577482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.577490 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.577894 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.577897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.578457 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.578458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.579128 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:46.579129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:46.579461 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:46.579462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:46.579556 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:46.579920 (+ 364us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:46.579921 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.579942 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:46.579991 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:46.580135 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:46.580137 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:46.580139 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:46.580188 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:08:46.580256 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":111,"threads_started":1}
I20260923 22:08:46.580727 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11205
I20260923 22:08:46.580829 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:46.580983 1670 external_mini_cluster.cc:1428] Setting key 1dc0beac08aff68d5cf0b8c382743c06
I20260923 22:08:46.581074 11333 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36659
I20260923 22:08:46.581204 11333 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:46.581378 11333 heartbeater.cc:511] Master 127.1.161.190:36659 requested a full tablet report, sending...
I20260923 22:08:46.581717 10883 ts_manager.cc:194] Registered new tserver with Master: f6416b15d11a47ccb2c8228f1c563ec4 (127.1.161.131:46681)
I20260923 22:08:46.582083 10883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.131:41761
I20260923 22:08:46.592656 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:08:46.594298 10926 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:46.593846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.1.161.190:36659)
0923 22:08:46.593894 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.593896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.593971 (+ 75us) server_negotiation.cc:407] Connection header received
0923 22:08:46.594034 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.594035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.594042 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.594081 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.594237 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54352: BlockingRecv error: recv got EOF from 127.0.0.1:54352 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:08:46.594370 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10943
I20260923 22:08:46.598662 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11074
I20260923 22:08:46.602764 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11205
I20260923 22:08:46.606314 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 10852
2026-09-23T22:08:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (531 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T22:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:46Z Disabled control of system clock
I20260923 22:08:46.630262 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44997
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:46.700095 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.700263 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.700284 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.701318 11344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:46.701351 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.701362 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.701373 11344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:46.701385 11344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:46.702478 11344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.702629 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.702817 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.706957 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.707002 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.707057 11344 server_base.cc:1034] running on GCE node
W20260923 22:08:46.707026 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.707345 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.707614 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.708753 11344 hybrid_clock.cc:648] HybridClock initialized: now 1790201326708750 us; error 34 us; skew 500 ppm
I20260923 22:08:46.709717 11344 webserver.cc:460] Webserver started at http://127.1.161.190:35721/ using document root <none> and password file <none>
I20260923 22:08:46.709949 11344 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.710045 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.710194 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.711153 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7efdee30d99a67013e34d1604dcf34fe"
server_key_iv: "73a5cd4199e1e8a7995d2e07e86508db"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.711515 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7efdee30d99a67013e34d1604dcf34fe"
server_key_iv: "73a5cd4199e1e8a7995d2e07e86508db"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.712561 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:46.713164 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.713299 11344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.713371 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "7efdee30d99a67013e34d1604dcf34fe"
server_key_iv: "73a5cd4199e1e8a7995d2e07e86508db"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.713428 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.716428 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.716598 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.716734 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.720222 11344 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:44997
I20260923 22:08:46.720250 11410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:44997 every 8 connection(s)
I20260923 22:08:46.720534 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:46.721088 11411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:46.723286 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f: Bootstrap starting.
I20260923 22:08:46.723536 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11344
I20260923 22:08:46.723628 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:46.723793 1670 external_mini_cluster.cc:1428] Setting key 54d7c41af3b04d2b141efb4a67e51ed4
I20260923 22:08:46.723817 11411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:46.724154 11411 log.cc:824] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:46.724841 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f: No bootstrap required, opened a new log
I20260923 22:08:46.726267 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.724234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.1.161.190:44997)
0923 22:08:46.724388 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.724396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.724501 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:08:46.725026 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.725034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.725076 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.725213 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.725893 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.725897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.725897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.725902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.725941 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.725953 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.726033 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.726059 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.726105 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260923 22:08:46.726466 11411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } }
I20260923 22:08:46.726591 11411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:46.726630 11411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955ed0fd7a664a1bb1ea1e6bc8690c9f, State: Initialized, Role: FOLLOWER
I20260923 22:08:46.726738 11411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } }
I20260923 22:08:46.726848 11411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:46.726887 11411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:46.726930 11411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:46.727712 11411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } }
I20260923 22:08:46.727828 11411 leader_election.cc:302] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 955ed0fd7a664a1bb1ea1e6bc8690c9f; no voters:
I20260923 22:08:46.728005 11411 leader_election.cc:288] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:46.728044 11416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:46.728201 11416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [term 1 LEADER]: Becoming Leader. State: Replica: 955ed0fd7a664a1bb1ea1e6bc8690c9f, State: Running, Role: LEADER
I20260923 22:08:46.728221 11411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:46.728293 11416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } }
I20260923 22:08:46.728632 11417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } } }
I20260923 22:08:46.728652 11418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 955ed0fd7a664a1bb1ea1e6bc8690c9f. Latest consensus state: current_term: 1 leader_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955ed0fd7a664a1bb1ea1e6bc8690c9f" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44997 } } }
I20260923 22:08:46.728724 11417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:46.728729 11418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:46.729106 11425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:46.729471 11425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:46.730788 11425 catalog_manager.cc:1347] Generated new cluster ID: 553b96e64b734487a5813c2a00622f35
I20260923 22:08:46.730839 11425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:08:46.737442 11425 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:46.737577 11425 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:46.747121 11425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 955ed0fd7a664a1bb1ea1e6bc8690c9f: Generated new TSK 0
I20260923 22:08:46.747278 11425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:08:46.752017 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--builtin_ntp_servers=127.1.161.148:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:46.823593 11435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.823762 11435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.823784 11435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.824888 11435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.824925 11435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.824949 11435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:46.826236 11435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.826442 11435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.826712 11435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:08:46.832346 11435 server_base.cc:1034] running on GCE node
W20260923 22:08:46.832322 11441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.832305 11440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.832540 11443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.832728 11435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.832937 11435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.834070 11435 hybrid_clock.cc:648] HybridClock initialized: now 1790201326834063 us; error 23 us; skew 500 ppm
I20260923 22:08:46.835115 11435 webserver.cc:460] Webserver started at http://127.1.161.129:44525/ using document root <none> and password file <none>
I20260923 22:08:46.835355 11435 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.835393 11435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.835510 11435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.836444 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "d969a109ef3e42598112995e6903647e"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "63542d8d457de79ab55de63ce94507fc"
server_key_iv: "42b291602cac505f30cfb1da1aae919c"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.836741 11435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "d969a109ef3e42598112995e6903647e"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "63542d8d457de79ab55de63ce94507fc"
server_key_iv: "42b291602cac505f30cfb1da1aae919c"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.838007 11435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:46.838703 11449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.838889 11435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.838977 11435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d969a109ef3e42598112995e6903647e"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "63542d8d457de79ab55de63ce94507fc"
server_key_iv: "42b291602cac505f30cfb1da1aae919c"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.839052 11435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.842111 11435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.842331 11435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.842449 11435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.842633 11435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:46.842886 11435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:46.842932 11435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.842967 11435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:46.842991 11435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.848246 11435 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37861
I20260923 22:08:46.848282 11562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37861 every 8 connection(s)
I20260923 22:08:46.848531 11435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:08:46.850858 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.849059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:53539 (local address 127.1.161.190:44997)
0923 22:08:46.849117 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.849119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.849338 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:08:46.849665 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.849666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.849670 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.849733 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.850683 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.850684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.850684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.850688 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.850701 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.850705 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.850749 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.850800 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.850811 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:46.851013 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.849071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44997 (local address 127.1.161.129:53539)
0923 22:08:46.849242 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.849267 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.849599 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.849773 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.849776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.849782 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.850552 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:46.850552 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.850583 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:46.850607 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:46.850747 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:46.850749 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:46.850751 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:46.850799 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 22:08:46.850838 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":93,"threads_started":1}
I20260923 22:08:46.851558 11563 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44997
I20260923 22:08:46.851636 11563 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:46.851763 11563 heartbeater.cc:511] Master 127.1.161.190:44997 requested a full tablet report, sending...
I20260923 22:08:46.852150 11375 ts_manager.cc:194] Registered new tserver with Master: d969a109ef3e42598112995e6903647e (127.1.161.129:37861)
I20260923 22:08:46.856261 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11435
I20260923 22:08:46.856364 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:46.856525 1670 external_mini_cluster.cc:1428] Setting key 497e07a76f57cdb09f77cc16c36f2dd6
I20260923 22:08:46.857267 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--builtin_ntp_servers=127.1.161.148:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 22:08:46.927040 11566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:46.927270 11566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:46.927295 11566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:46.928402 11566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:46.928436 11566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:46.928459 11566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:46.929749 11566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:46.929947 11566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:46.930231 11566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:46.934513 11571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.934522 11574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:46.934522 11572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:46.934595 11566 server_base.cc:1034] running on GCE node
I20260923 22:08:46.934891 11566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:46.935092 11566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:46.936249 11566 hybrid_clock.cc:648] HybridClock initialized: now 1790201326936242 us; error 27 us; skew 500 ppm
I20260923 22:08:46.937312 11566 webserver.cc:460] Webserver started at http://127.1.161.130:37099/ using document root <none> and password file <none>
I20260923 22:08:46.937531 11566 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:46.937600 11566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:46.937708 11566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:46.938608 11566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "6c2f2b4e041142e7a28ac483173e9638"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "4652d0f1279446806f327a3c61071d61"
server_key_iv: "872c7bb7bdb11a5806eec92f16ea710e"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.938913 11566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "6c2f2b4e041142e7a28ac483173e9638"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "4652d0f1279446806f327a3c61071d61"
server_key_iv: "872c7bb7bdb11a5806eec92f16ea710e"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.940148 11566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:46.940835 11580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.940984 11566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:46.941083 11566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "6c2f2b4e041142e7a28ac483173e9638"
format_stamp: "Formatted at 2026-09-23 22:08:46 on dist-test-slave-npvd"
server_key: "4652d0f1279446806f327a3c61071d61"
server_key_iv: "872c7bb7bdb11a5806eec92f16ea710e"
server_key_version: "encryptionkey@0"
I20260923 22:08:46.941197 11566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:46.944227 11566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:46.944388 11566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:46.944464 11566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:46.944593 11566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:46.944780 11566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:46.944805 11566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.944821 11566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:46.944832 11566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:46.949456 11566 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:45985
I20260923 22:08:46.949517 11693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:45985 every 8 connection(s)
I20260923 22:08:46.949750 11566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:08:46.951028 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11566
I20260923 22:08:46.951128 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:46.951380 1670 external_mini_cluster.cc:1428] Setting key 6c78fadb0dbe6caa451850164b2d374b
I20260923 22:08:46.952224 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--builtin_ntp_servers=127.1.161.148:35933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 22:08:46.952360 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.950381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:57247 (local address 127.1.161.190:44997)
0923 22:08:46.950452 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:46.950454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:46.950616 (+ 162us) server_negotiation.cc:407] Connection header received
0923 22:08:46.950974 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:46.950976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:46.950982 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:46.951056 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:46.952140 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:46.952141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:46.952141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:46.952145 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:46.952160 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:46.952165 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:46.952225 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:46.952268 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:08:46.952281 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:08:46.952515 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:46.950346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44997 (local address 127.1.161.130:57247)
0923 22:08:46.950512 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:46.950540 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:46.950891 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:46.951104 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:46.951107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:46.951114 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:46.952003 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:46.952004 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:46.952032 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:46.952057 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:46.952222 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:46.952224 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:46.952226 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:08:46.952285 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:08:46.952337 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260923 22:08:46.953614 11694 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44997
I20260923 22:08:46.953701 11694 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:46.953817 11694 heartbeater.cc:511] Master 127.1.161.190:44997 requested a full tablet report, sending...
I20260923 22:08:46.954120 11375 ts_manager.cc:194] Registered new tserver with Master: 6c2f2b4e041142e7a28ac483173e9638 (127.1.161.130:45985)
W20260923 22:08:47.022011 11697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:47.022171 11697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:47.022190 11697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:47.023083 11697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:47.023120 11697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:47.023142 11697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:47.024293 11697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:35933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:47.024446 11697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:47.024621 11697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:47.028967 11703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:47.028982 11705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:47.028981 11702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:47.029322 11697 server_base.cc:1034] running on GCE node
I20260923 22:08:47.029487 11697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:47.029697 11697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:47.030827 11697 hybrid_clock.cc:648] HybridClock initialized: now 1790201327030818 us; error 32 us; skew 500 ppm
I20260923 22:08:47.031946 11697 webserver.cc:460] Webserver started at http://127.1.161.131:46761/ using document root <none> and password file <none>
I20260923 22:08:47.032166 11697 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:47.032232 11697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:47.032378 11697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:47.033295 11697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "ac503e6c45104a4ea842b6e8611b238b"
format_stamp: "Formatted at 2026-09-23 22:08:47 on dist-test-slave-npvd"
server_key: "3460a09bd75eb7a36112e493db3c6f09"
server_key_iv: "27a5e98989e9ec020864ea864ebde728"
server_key_version: "encryptionkey@0"
I20260923 22:08:47.033608 11697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "ac503e6c45104a4ea842b6e8611b238b"
format_stamp: "Formatted at 2026-09-23 22:08:47 on dist-test-slave-npvd"
server_key: "3460a09bd75eb7a36112e493db3c6f09"
server_key_iv: "27a5e98989e9ec020864ea864ebde728"
server_key_version: "encryptionkey@0"
I20260923 22:08:47.034713 11697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:08:47.035435 11711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:47.035580 11697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:08:47.035668 11697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "ac503e6c45104a4ea842b6e8611b238b"
format_stamp: "Formatted at 2026-09-23 22:08:47 on dist-test-slave-npvd"
server_key: "3460a09bd75eb7a36112e493db3c6f09"
server_key_iv: "27a5e98989e9ec020864ea864ebde728"
server_key_version: "encryptionkey@0"
I20260923 22:08:47.035739 11697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:47.038477 11697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:47.038691 11697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:47.038823 11697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:47.039024 11697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:47.039330 11697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:47.039386 11697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:47.039418 11697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:47.039441 11697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:47.044845 11697 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:36749
I20260923 22:08:47.044917 11824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:36749 every 8 connection(s)
I20260923 22:08:47.045156 11697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:08:47.046151 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11697
I20260923 22:08:47.046231 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:47.046407 1670 external_mini_cluster.cc:1428] Setting key 1e4a8ab1fd749d894b38ceb9f1164523
I20260923 22:08:47.047660 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:47.045699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:50587 (local address 127.1.161.190:44997)
0923 22:08:47.045760 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:47.045762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:47.045911 (+ 149us) server_negotiation.cc:407] Connection header received
0923 22:08:47.046409 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:47.046410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:47.046414 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:47.046478 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:47.047500 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:47.047500 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:47.047501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:08:47.047503 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:47.047515 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:08:47.047525 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:47.047557 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:47.047608 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 22:08:47.047617 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:08:47.047818 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:47.045686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44997 (local address 127.1.161.131:50587)
0923 22:08:47.045824 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:47.045852 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:47.046314 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:47.046574 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:47.046577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:47.046585 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:47.047393 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:08:47.047394 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:47.047414 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:08:47.047434 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:47.047557 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:47.047559 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:47.047560 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:08:47.047612 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:08:47.047646 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260923 22:08:47.048172 11825 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44997
I20260923 22:08:47.048245 11825 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:47.048332 11825 heartbeater.cc:511] Master 127.1.161.190:44997 requested a full tablet report, sending...
I20260923 22:08:47.048574 11375 ts_manager.cc:194] Registered new tserver with Master: ac503e6c45104a4ea842b6e8611b238b (127.1.161.131:36749)
I20260923 22:08:47.058163 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 22:08:47.059621 11414 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:08:47.059200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.1.161.190:44997)
0923 22:08:47.059253 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:47.059255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:47.059348 (+ 93us) server_negotiation.cc:407] Connection header received
0923 22:08:47.059411 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:47.059412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:47.059416 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:47.059458 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:47.059562 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45032: BlockingRecv error: recv got EOF from 127.0.0.1:45032 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:08:47.059674 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11435
I20260923 22:08:47.063580 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11566
I20260923 22:08:47.067011 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11697
I20260923 22:08:47.070497 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11344
2026-09-23T22:08:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (464 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:47 dist-test-slave-npvd krb5kdc[11835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:47 dist-test-slave-npvd krb5kdc[11835](info): set up 2 sockets
Sep 23 22:08:47 dist-test-slave-npvd krb5kdc[11835](info): commencing operation
krb5kdc: starting...
W20260923 22:08:47.987370 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.899s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:08:48.086822 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41857
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:48.158757 11848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:48.158958 11848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:48.158988 11848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:48.159965 11848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:08:48.160022 11848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:48.160043 11848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:48.160063 11848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:08:48.160089 11848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:08:48.161490 11848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:48.161725 11848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:48.161911 11848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:48.165859 11854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.165879 11853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.165902 11848 server_base.cc:1034] running on GCE node
W20260923 22:08:48.165982 11856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.166235 11848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:48.166486 11848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:48.167631 11848 hybrid_clock.cc:648] HybridClock initialized: now 1790201328167617 us; error 23 us; skew 500 ppm
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:48.177102 11848 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:08:48.177450 11848 webserver.cc:460] Webserver started at http://127.1.161.190:42605/ using document root <none> and password file <none>
I20260923 22:08:48.177644 11848 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:48.177702 11848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:48.177829 11848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:48.178676 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "b0a88fa38cb841399e2a8b81c383c8ce"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "145eef2a5d91071f6c4089a90a1a9e8f"
server_key_iv: "35f12251ab39f9af69a27947850b1e89"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.178977 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "b0a88fa38cb841399e2a8b81c383c8ce"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "145eef2a5d91071f6c4089a90a1a9e8f"
server_key_iv: "35f12251ab39f9af69a27947850b1e89"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.180130 11848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:48.180821 11863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.181114 11848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:08:48.181210 11848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "b0a88fa38cb841399e2a8b81c383c8ce"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "145eef2a5d91071f6c4089a90a1a9e8f"
server_key_iv: "35f12251ab39f9af69a27947850b1e89"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.181288 11848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:48.188723 11848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:48.190584 11848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:48.190742 11848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:48.194047 11848 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41857
I20260923 22:08:48.194074 11915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41857 every 8 connection(s)
I20260923 22:08:48.194339 11848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:08:48.195127 11916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:48.197067 11916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: Bootstrap starting.
I20260923 22:08:48.197616 11916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:48.197889 11916 log.cc:824] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:48.198541 11916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: No bootstrap required, opened a new log
I20260923 22:08:48.199971 11916 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } }
I20260923 22:08:48.200155 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11848
I20260923 22:08:48.200150 11916 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:48.200199 11916 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a88fa38cb841399e2a8b81c383c8ce, State: Initialized, Role: FOLLOWER
I20260923 22:08:48.200265 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:08:48.200364 11916 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } }
I20260923 22:08:48.200433 1670 external_mini_cluster.cc:1428] Setting key 3e74c50077bb2d35466aa3832030b4a5
I20260923 22:08:48.200440 11916 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:08:48.200475 11916 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:08:48.200526 11916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:48.201203 11916 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } }
I20260923 22:08:48.201306 11916 leader_election.cc:302] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0a88fa38cb841399e2a8b81c383c8ce; no voters:
I20260923 22:08:48.201479 11916 leader_election.cc:288] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:08:48.201499 11921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:48.201701 11921 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [term 1 LEADER]: Becoming Leader. State: Replica: b0a88fa38cb841399e2a8b81c383c8ce, State: Running, Role: LEADER
I20260923 22:08:48.201705 11916 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:08:48.201792 11921 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } }
I20260923 22:08:48.202055 11923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0a88fa38cb841399e2a8b81c383c8ce. Latest consensus state: current_term: 1 leader_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } } }
I20260923 22:08:48.202133 11923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:48.202335 11925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:08:48.202878 11925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:08:48.203333 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a88fa38cb841399e2a8b81c383c8ce" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41857 } } }
I20260923 22:08:48.203395 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce [sys.catalog]: This master's current role is: LEADER
I20260923 22:08:48.204384 11925 catalog_manager.cc:1347] Generated new cluster ID: 4670c56507ec4b8480278364e1514515
I20260923 22:08:48.204424 11925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:48.211508 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.200876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.1.161.190:41857)
0923 22:08:48.201369 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.201377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.201425 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:08:48.201551 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.201557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.201623 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.202468 (+ 845us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:48.202548 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.203853 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.204735 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.205227 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.205316 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.207640 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:48.207643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:48.207644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:48.207654 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:48.209901 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.210292 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.210295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.210296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.210413 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.210609 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.210610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.210610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.210907 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:48.211063 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.211127 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.211289 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":395,"threads_started":1}
I20260923 22:08:48.221172 11925 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:08:48.221832 11925 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:08:48.235512 11925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: Generated new TSK 0
I20260923 22:08:48.235672 11925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:08:48.259331 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:48.328446 11942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:48.328613 11942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:48.328634 11942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:48.329661 11942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:48.329697 11942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:48.329721 11942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:08:48.331180 11942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:48.331398 11942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:48.331645 11942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:48.335871 11948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.335883 11950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.335915 11947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.336066 11942 server_base.cc:1034] running on GCE node
I20260923 22:08:48.336254 11942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:48.336460 11942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:48.337626 11942 hybrid_clock.cc:648] HybridClock initialized: now 1790201328337611 us; error 28 us; skew 500 ppm
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:48.340175 11942 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:08:48.340602 11942 webserver.cc:460] Webserver started at http://127.1.161.129:33715/ using document root <none> and password file <none>
I20260923 22:08:48.340826 11942 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:48.340895 11942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:48.341048 11942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:48.341970 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "5677e25094284de5a2f38cebe0ce5124"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "4a1132afa4991c8f7894f2879be3f50e"
server_key_iv: "74cf83e25bc981fb9f5226633c51e693"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.342325 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "5677e25094284de5a2f38cebe0ce5124"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "4a1132afa4991c8f7894f2879be3f50e"
server_key_iv: "74cf83e25bc981fb9f5226633c51e693"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.343621 11942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:08:48.344249 11957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.344414 11942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:48.344506 11942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "5677e25094284de5a2f38cebe0ce5124"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "4a1132afa4991c8f7894f2879be3f50e"
server_key_iv: "74cf83e25bc981fb9f5226633c51e693"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.344583 11942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:48.356557 11942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:48.358705 11942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:48.358850 11942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:48.359040 11942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:48.359341 11942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:48.359398 11942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.359436 11942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:48.359462 11942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.365234 11942 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46091
I20260923 22:08:48.365298 12070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46091 every 8 connection(s)
I20260923 22:08:48.365554 11942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:48.373082 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 11942
I20260923 22:08:48.373129 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.366183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:53035 (local address 127.1.161.190:41857)
0923 22:08:48.366228 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.366230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.366420 (+ 190us) server_negotiation.cc:407] Connection header received
0923 22:08:48.366787 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.366788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.366796 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.366839 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:48.367631 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.368046 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.368857 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.369012 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.369054 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.370931 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:48.370932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:48.370933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:48.370939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:48.372071 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.372392 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.372393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.372393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.372434 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.372616 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.372616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.372616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.372704 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:48.372744 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.372971 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.373043 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:08:48.373163 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:08:48.373188 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.366158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41857 (local address 127.1.161.129:53035)
0923 22:08:48.366313 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.366341 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.366715 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.366881 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.366884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.367404 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:48.367556 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.367560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.368137 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.368138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.368788 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.368790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.369075 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.369075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.369179 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.369616 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:48.369623 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:48.370839 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:48.372144 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.372149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.372150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.372331 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.372486 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.372487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.372487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.372567 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.372753 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:48.372757 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:48.372843 (+ 86us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.372913 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.373008 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260923 22:08:48.373337 1670 external_mini_cluster.cc:1428] Setting key 603b18858eb336a552bed8adb1c9df24
I20260923 22:08:48.373624 12071 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41857
I20260923 22:08:48.373715 12071 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:48.373857 12071 heartbeater.cc:511] Master 127.1.161.190:41857 requested a full tablet report, sending...
I20260923 22:08:48.374429 11880 ts_manager.cc:194] Registered new tserver with Master: 5677e25094284de5a2f38cebe0ce5124 (127.1.161.129:46091)
I20260923 22:08:48.375362 11880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:53035
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:08:48.394811 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:48.463709 12076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:48.463896 12076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:48.463927 12076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:48.464876 12076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:48.464932 12076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:48.464970 12076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:08:48.466146 12076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:48.466373 12076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:48.466550 12076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:48.470647 12081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.470666 12082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.470708 12076 server_base.cc:1034] running on GCE node
W20260923 22:08:48.470691 12084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.471016 12076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:48.471280 12076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:48.472442 12076 hybrid_clock.cc:648] HybridClock initialized: now 1790201328472428 us; error 25 us; skew 500 ppm
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:48.474462 12076 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:08:48.474795 12076 webserver.cc:460] Webserver started at http://127.1.161.130:40745/ using document root <none> and password file <none>
I20260923 22:08:48.474974 12076 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:48.475029 12076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:48.475188 12076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:48.475976 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "edfa6ce1d0c94b099858a79c02a2847e"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "5cdcf71fce8d39fe81b0b121966078c8"
server_key_iv: "13c3f89468e141d8e2b92875736d15f0"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.476281 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "edfa6ce1d0c94b099858a79c02a2847e"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "5cdcf71fce8d39fe81b0b121966078c8"
server_key_iv: "13c3f89468e141d8e2b92875736d15f0"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.477342 12076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:08:48.477901 12091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.478030 12076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:48.478107 12076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "edfa6ce1d0c94b099858a79c02a2847e"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "5cdcf71fce8d39fe81b0b121966078c8"
server_key_iv: "13c3f89468e141d8e2b92875736d15f0"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.478189 12076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:48.492170 12076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:48.493989 12076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:48.494120 12076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:48.494306 12076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:48.494545 12076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:48.494590 12076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.494623 12076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:48.494650 12076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.499536 12076 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:43155
I20260923 22:08:48.499627 12204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:43155 every 8 connection(s)
I20260923 22:08:48.499842 12076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:48.507328 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.500456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:56031 (local address 127.1.161.190:41857)
0923 22:08:48.500495 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.500497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.500702 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:08:48.501126 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.501127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.501136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.501184 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:48.502026 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.502431 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.503396 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.503554 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.503602 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.505243 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:48.505244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:48.505245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:48.505250 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:48.506397 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.506672 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.506672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.506673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.506714 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.506873 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.506874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.506874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.506948 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:48.506984 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.507205 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.507275 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:48.507452 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.500426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41857 (local address 127.1.161.130:56031)
0923 22:08:48.500564 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.500601 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.501048 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.501237 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.501240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.501819 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:48.501964 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.501967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.502499 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.502500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.503336 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.503338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.503614 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.503615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.503696 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.504105 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:48.504111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:48.505174 (+ 1063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:48.506455 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.506458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.506460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.506620 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.506762 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.506763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.506763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.506837 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.506989 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:48.506992 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:48.507074 (+ 82us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.507144 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.507286 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260923 22:08:48.507853 12205 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41857
I20260923 22:08:48.507932 12205 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:48.508033 12205 heartbeater.cc:511] Master 127.1.161.190:41857 requested a full tablet report, sending...
I20260923 22:08:48.508338 11880 ts_manager.cc:194] Registered new tserver with Master: edfa6ce1d0c94b099858a79c02a2847e (127.1.161.130:43155)
I20260923 22:08:48.508366 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 12076
I20260923 22:08:48.508441 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:08:48.508610 1670 external_mini_cluster.cc:1428] Setting key 76f6dd35e4a713d4ab9a9b0bbc4a52e2
I20260923 22:08:48.508757 11880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:56031
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:08:48.529773 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:08:48.600469 12210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:08:48.600677 12210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:08:48.600714 12210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:08:48.601909 12210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:08:48.601961 12210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:08:48.602000 12210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:08:48.603395 12210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:08:48.603613 12210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:08:48.603819 12210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:08:48.608232 12218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.608232 12215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:08:48.608250 12216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:08:48.608279 12210 server_base.cc:1034] running on GCE node
I20260923 22:08:48.608541 12210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:08:48.608763 12210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:08:48.609910 12210 hybrid_clock.cc:648] HybridClock initialized: now 1790201328609956 us; error 82 us; skew 500 ppm
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:08:48.612191 12210 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:08:48.612524 12210 webserver.cc:460] Webserver started at http://127.1.161.131:40665/ using document root <none> and password file <none>
I20260923 22:08:48.612680 12210 fs_manager.cc:359] Metadata directory not provided
I20260923 22:08:48.612725 12210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:08:48.612800 12210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:08:48.613704 12210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "e3b2cf8cbf734b48a2e06aac88b989fd"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "db6d0fb4974477db3f5c1ce5cef875f6"
server_key_iv: "04993df9dac84a46c65787aebc4ef1e1"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.613970 12210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "e3b2cf8cbf734b48a2e06aac88b989fd"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "db6d0fb4974477db3f5c1ce5cef875f6"
server_key_iv: "04993df9dac84a46c65787aebc4ef1e1"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.615103 12210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:08:48.615789 12225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.615983 12210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:08:48.616081 12210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "e3b2cf8cbf734b48a2e06aac88b989fd"
format_stamp: "Formatted at 2026-09-23 22:08:48 on dist-test-slave-npvd"
server_key: "db6d0fb4974477db3f5c1ce5cef875f6"
server_key_iv: "04993df9dac84a46c65787aebc4ef1e1"
server_key_version: "encryptionkey@0"
I20260923 22:08:48.616158 12210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:08:48.626940 12210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:08:48.628922 12210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:08:48.629099 12210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:08:48.629313 12210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:08:48.629583 12210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:08:48.629634 12210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.629673 12210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:08:48.629700 12210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:08:48.635386 12210 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:34527
I20260923 22:08:48.635427 12338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:34527 every 8 connection(s)
I20260923 22:08:48.635699 12210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[11835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201328, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:08:48.643708 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 12210
I20260923 22:08:48.643836 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:08:48.644037 1670 external_mini_cluster.cc:1428] Setting key f147259ebd6e5df1157636cfe4d25fdc
I20260923 22:08:48.644459 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.636285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:51747 (local address 127.1.161.190:41857)
0923 22:08:48.636352 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.636354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.636498 (+ 144us) server_negotiation.cc:407] Connection header received
0923 22:08:48.636956 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.636957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.636967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.637027 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:48.637850 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.638337 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.639295 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.639487 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.639544 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.641464 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:48.641466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:48.641466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:48.641473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:48.642980 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.643450 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.643451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.643452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.643533 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.643763 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.643764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.643764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.643910 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:48.643992 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.644234 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.644328 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 22:08:48.644647 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.636239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41857 (local address 127.1.161.131:51747)
0923 22:08:48.636377 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.636413 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.636841 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.637080 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.637083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.637600 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:08:48.637770 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.637774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.638413 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.638413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.639159 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.639160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.639558 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.639559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.639651 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.640056 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:08:48.640063 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:08:48.641374 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:08:48.643069 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.643074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.643075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.643365 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.643612 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:08:48.643612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:08:48.643612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:08:48.643696 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:08:48.643981 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:08:48.643985 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:08:48.644103 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.644221 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.644423 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260923 22:08:48.645143 12339 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41857
I20260923 22:08:48.645238 12339 heartbeater.cc:464] Registering TS with master...
I20260923 22:08:48.645402 12339 heartbeater.cc:511] Master 127.1.161.190:41857 requested a full tablet report, sending...
I20260923 22:08:48.645744 11880 ts_manager.cc:194] Registered new tserver with Master: e3b2cf8cbf734b48a2e06aac88b989fd (127.1.161.131:34527)
I20260923 22:08:48.646045 11880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:51747
I20260923 22:08:48.655726 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:08:48.662010 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.656903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.1.161.190:41857)
0923 22:08:48.656958 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.656960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.657081 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:08:48.657112 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.657113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.657120 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.657160 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:08:48.657628 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.658039 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.658661 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.658818 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.658869 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.659859 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:08:48.659860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:08:48.659861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:08:48.659866 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:08:48.661196 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.661459 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.661460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.661460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.661516 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:08:48.661643 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:08:48.661644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:08:48.661644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:08:48.661749 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:08:48.661810 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.661891 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.661939 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:08:48.663544 11880 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:08:48.664927 11880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:08:48.670706 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.666842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.1.161.130:43155)
0923 22:08:48.667005 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.667010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.667045 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:08:48.667083 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.667085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.667099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.667192 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:48.667361 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668511 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.669765 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670274 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670387 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670440 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.670589 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.670626 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260923 22:08:48.670714 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.666758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:43155 (local address 127.0.0.1:51816)
0923 22:08:48.666906 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.666939 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.666993 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.667230 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.667233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.667235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:48.667288 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.667290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668626 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.668626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.669684 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.669686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670399 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.670481 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670485 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.670574 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.670616 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":25,"thread_start_us":53,"threads_started":1}
I20260923 22:08:48.670717 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.666758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:34527 (local address 127.0.0.1:51152)
0923 22:08:48.667044 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.667055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.667099 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.667567 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.667568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.667570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:48.667610 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.667611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668412 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.668413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.669589 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.669592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670391 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.670470 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670482 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.670528 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.670582 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260923 22:08:48.670876 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.666833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.1.161.131:34527)
0923 22:08:48.667400 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.667403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.667409 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:08:48.667431 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.667432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.667461 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.667559 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:48.667664 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668320 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.669669 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670286 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670663 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670720 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.670747 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.670787 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":70,"threads_started":1}
I20260923 22:08:48.670938 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.666933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46091 (local address 127.0.0.1:58112)
0923 22:08:48.667078 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.667086 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.667123 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.667482 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.667483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.667485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:48.667522 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.667524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668742 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.668743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.669494 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.669496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670446 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.670496 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670499 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.670578 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.670610 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260923 22:08:48.671356 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.667135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.1.161.129:46091)
0923 22:08:48.667313 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.667316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.667342 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:08:48.667375 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.667377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.667392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.667469 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:48.667693 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.668637 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.669564 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.670272 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.670345 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.670418 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.671241 (+ 823us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.671282 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260923 22:08:48.671582 12273 tablet_service.cc:1467] Processing CreateTablet for tablet f645844a9a404035bb0962b989a9a33a (DEFAULT_TABLE table=test-table [id=2a70b84d1ebb413388a5ff30da154165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:48.671904 12273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f645844a9a404035bb0962b989a9a33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:48.671882 12139 tablet_service.cc:1467] Processing CreateTablet for tablet f645844a9a404035bb0962b989a9a33a (DEFAULT_TABLE table=test-table [id=2a70b84d1ebb413388a5ff30da154165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:48.672030 12005 tablet_service.cc:1467] Processing CreateTablet for tablet f645844a9a404035bb0962b989a9a33a (DEFAULT_TABLE table=test-table [id=2a70b84d1ebb413388a5ff30da154165]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:08:48.672178 12139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f645844a9a404035bb0962b989a9a33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:48.672360 12005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f645844a9a404035bb0962b989a9a33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:08:48.674587 12353 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: Bootstrap starting.
I20260923 22:08:48.675100 12354 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Bootstrap starting.
I20260923 22:08:48.675105 12355 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Bootstrap starting.
I20260923 22:08:48.675643 12353 tablet_bootstrap.cc:654] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:48.675660 12354 tablet_bootstrap.cc:654] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:48.675868 12355 tablet_bootstrap.cc:654] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Neither blocks nor log segments found. Creating new log.
I20260923 22:08:48.676159 12354 log.cc:824] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:48.676260 12353 log.cc:824] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:48.676395 12355 log.cc:824] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:08:48.676898 12354 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: No bootstrap required, opened a new log
I20260923 22:08:48.676940 12353 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: No bootstrap required, opened a new log
I20260923 22:08:48.676978 12354 ts_tablet_manager.cc:1397] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:48.676987 12353 ts_tablet_manager.cc:1397] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:48.677002 12355 tablet_bootstrap.cc:492] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: No bootstrap required, opened a new log
I20260923 22:08:48.677044 12355 ts_tablet_manager.cc:1397] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:08:48.678251 12355 raft_consensus.cc:348] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.678382 12355 raft_consensus.cc:374] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:48.678417 12355 raft_consensus.cc:729] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edfa6ce1d0c94b099858a79c02a2847e, State: Initialized, Role: FOLLOWER
I20260923 22:08:48.678521 12355 consensus_queue.cc:260] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.678793 12355 ts_tablet_manager.cc:1428] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:48.678889 12205 heartbeater.cc:503] Master 127.1.161.190:41857 was elected leader, sending a full tablet report...
I20260923 22:08:48.678859 12353 raft_consensus.cc:348] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.678859 12354 raft_consensus.cc:348] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.678996 12354 raft_consensus.cc:374] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:48.678995 12353 raft_consensus.cc:374] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:08:48.679028 12353 raft_consensus.cc:729] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5677e25094284de5a2f38cebe0ce5124, State: Initialized, Role: FOLLOWER
I20260923 22:08:48.679028 12354 raft_consensus.cc:729] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b2cf8cbf734b48a2e06aac88b989fd, State: Initialized, Role: FOLLOWER
I20260923 22:08:48.679126 12354 consensus_queue.cc:260] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.679126 12353 consensus_queue.cc:260] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.679427 12354 ts_tablet_manager.cc:1428] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:08:48.679427 12353 ts_tablet_manager.cc:1428] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:08:48.679544 12339 heartbeater.cc:503] Master 127.1.161.190:41857 was elected leader, sending a full tablet report...
I20260923 22:08:48.679581 12071 heartbeater.cc:503] Master 127.1.161.190:41857 was elected leader, sending a full tablet report...
I20260923 22:08:48.730844 12360 raft_consensus.cc:482] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:08:48.731058 12360 raft_consensus.cc:504] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.731432 12360 leader_election.cc:288] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edfa6ce1d0c94b099858a79c02a2847e (127.1.161.130:43155), e3b2cf8cbf734b48a2e06aac88b989fd (127.1.161.131:34527)
I20260923 22:08:48.734066 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.731672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:36731 (local address 127.1.161.130:43155)
0923 22:08:48.731707 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.731709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.731716 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:08:48.731739 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.731740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.731748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.731820 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:48.731957 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.732439 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.733358 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.733716 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.733790 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.733798 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.733973 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.734006 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:08:48.734067 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.731585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:43155 (local address 127.1.161.129:36731)
0923 22:08:48.731632 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.731643 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.731682 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.731823 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.731824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.731825 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:48.731890 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.731892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.732526 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.732526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.733266 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.733268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.733801 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.733802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.733873 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.733894 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.733966 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.734005 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:08:48.734330 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.731681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:59097 (local address 127.1.161.131:34527)
0923 22:08:48.731719 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.731721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.731859 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:08:48.731927 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.731927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.731935 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.731995 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:08:48.732147 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.732573 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.733534 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.733946 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.734031 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.734038 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.734235 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.734278 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:48.734337 12159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f645844a9a404035bb0962b989a9a33a" candidate_uuid: "5677e25094284de5a2f38cebe0ce5124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfa6ce1d0c94b099858a79c02a2847e" is_pre_election: true
I20260923 22:08:48.734498 12159 raft_consensus.cc:2454] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5677e25094284de5a2f38cebe0ce5124 in term 0.
I20260923 22:08:48.734537 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.731624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:34527 (local address 127.1.161.129:59097)
0923 22:08:48.731788 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.731796 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:08:48.731866 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:08:48.732013 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:08:48.732013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:08:48.732015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:08:48.732080 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.732082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.732669 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.732670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.733447 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:08:48.733449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.734033 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.734034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:08:48.734104 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.734125 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 22:08:48.734422 (+ 297us) client_negotiation.cc:240] Negotiation successful
0923 22:08:48.734468 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260923 22:08:48.734747 11958 leader_election.cc:302] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5677e25094284de5a2f38cebe0ce5124, edfa6ce1d0c94b099858a79c02a2847e; no voters:
I20260923 22:08:48.734886 12360 raft_consensus.cc:2781] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:08:48.734871 12293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f645844a9a404035bb0962b989a9a33a" candidate_uuid: "5677e25094284de5a2f38cebe0ce5124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" is_pre_election: true
I20260923 22:08:48.734953 12360 raft_consensus.cc:482] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:08:48.734970 12360 raft_consensus.cc:3037] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:48.735006 12293 raft_consensus.cc:2454] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5677e25094284de5a2f38cebe0ce5124 in term 0.
I20260923 22:08:48.735697 12360 raft_consensus.cc:504] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.735827 12360 leader_election.cc:288] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [CANDIDATE]: Term 1 election: Requested vote from peers edfa6ce1d0c94b099858a79c02a2847e (127.1.161.130:43155), e3b2cf8cbf734b48a2e06aac88b989fd (127.1.161.131:34527)
I20260923 22:08:48.735972 12293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f645844a9a404035bb0962b989a9a33a" candidate_uuid: "5677e25094284de5a2f38cebe0ce5124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd"
I20260923 22:08:48.736045 12293 raft_consensus.cc:3037] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:48.736079 12159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f645844a9a404035bb0962b989a9a33a" candidate_uuid: "5677e25094284de5a2f38cebe0ce5124" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edfa6ce1d0c94b099858a79c02a2847e"
I20260923 22:08:48.736152 12159 raft_consensus.cc:3037] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:08:48.736680 12293 raft_consensus.cc:2454] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5677e25094284de5a2f38cebe0ce5124 in term 1.
I20260923 22:08:48.736799 12159 raft_consensus.cc:2454] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5677e25094284de5a2f38cebe0ce5124 in term 1.
I20260923 22:08:48.736845 11960 leader_election.cc:302] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5677e25094284de5a2f38cebe0ce5124, e3b2cf8cbf734b48a2e06aac88b989fd; no voters:
I20260923 22:08:48.736959 12360 raft_consensus.cc:2781] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:08:48.737111 12360 raft_consensus.cc:686] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 1 LEADER]: Becoming Leader. State: Replica: 5677e25094284de5a2f38cebe0ce5124, State: Running, Role: LEADER
I20260923 22:08:48.737211 12360 consensus_queue.cc:237] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } }
I20260923 22:08:48.737882 11880 catalog_manager.cc:5495] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5677e25094284de5a2f38cebe0ce5124 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "5677e25094284de5a2f38cebe0ce5124" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5677e25094284de5a2f38cebe0ce5124" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46091 } health_report { overall_health: HEALTHY } } }
I20260923 22:08:48.742173 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.740297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.1.161.129:46091)
0923 22:08:48.740331 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:08:48.740351 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 22:08:48.740359 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:08:48.740406 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:08:48.740407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:08:48.740419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:08:48.740467 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:08:48.740568 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.740971 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.741643 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:08:48.741805 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:08:48.741850 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:08:48.741954 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:08:48.742028 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:08:48.742077 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:08:48.742108 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 22:08:48.742119 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:08:48.743619 12293 raft_consensus.cc:1261] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 1 FOLLOWER]: Refusing update from remote peer 5677e25094284de5a2f38cebe0ce5124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:48.743621 12159 raft_consensus.cc:1261] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 1 FOLLOWER]: Refusing update from remote peer 5677e25094284de5a2f38cebe0ce5124: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:08:48.743884 12360 consensus_queue.cc:1035] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3b2cf8cbf734b48a2e06aac88b989fd" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:48.744024 12363 consensus_queue.cc:1035] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edfa6ce1d0c94b099858a79c02a2847e" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:08:48.746084 12366 mvcc.cc:205] Tried to move back new op lower bound from 7332664642531438592 to 7332664642508058624. Current Snapshot: MvccSnapshot[applied={T|T < 7332664642531438592}]
I20260923 22:08:48.747001 12367 mvcc.cc:205] Tried to move back new op lower bound from 7332664642531438592 to 7332664642508058624. Current Snapshot: MvccSnapshot[applied={T|T < 7332664642531438592 or (T in {7332664642531438592})}]
I20260923 22:08:48.748231 11880 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:48.748328 11880 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:08:48.748921 11880 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0a88fa38cb841399e2a8b81c383c8ce: Sending DeleteTablet for 3 replicas of tablet f645844a9a404035bb0962b989a9a33a
I20260923 22:08:48.749238 12273 tablet_service.cc:1514] Processing DeleteTablet for tablet f645844a9a404035bb0962b989a9a33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:48 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:51152
I20260923 22:08:48.749264 12005 tablet_service.cc:1514] Processing DeleteTablet for tablet f645844a9a404035bb0962b989a9a33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:48 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:58112
I20260923 22:08:48.749434 12377 tablet_replica.cc:330] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124: stopping tablet replica
I20260923 22:08:48.749431 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11942
I20260923 22:08:48.749436 12139 tablet_service.cc:1514] Processing DeleteTablet for tablet f645844a9a404035bb0962b989a9a33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:08:48 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:51816
I20260923 22:08:48.749511 12377 raft_consensus.cc:2229] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:08:48.749586 12377 raft_consensus.cc:2258] T f645844a9a404035bb0962b989a9a33a P 5677e25094284de5a2f38cebe0ce5124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:48.750082 12378 tablet_replica.cc:330] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: stopping tablet replica
I20260923 22:08:48.750191 12378 raft_consensus.cc:2229] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:48.750286 12378 raft_consensus.cc:2258] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:48.750564 12376 tablet_replica.cc:330] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: stopping tablet replica
I20260923 22:08:48.750667 12376 raft_consensus.cc:2229] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:08:48.750665 12378 ts_tablet_manager.cc:1905] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:48.750759 12376 raft_consensus.cc:2258] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:08:48.751087 12376 ts_tablet_manager.cc:1905] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:08:48.752238 12376 ts_tablet_manager.cc:1918] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:48.752316 12376 log.cc:1196] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/f645844a9a404035bb0962b989a9a33a
I20260923 22:08:48.752440 12378 ts_tablet_manager.cc:1918] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:08:48.752517 12378 log.cc:1196] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/f645844a9a404035bb0962b989a9a33a
I20260923 22:08:48.752574 12376 ts_tablet_manager.cc:1939] T f645844a9a404035bb0962b989a9a33a P e3b2cf8cbf734b48a2e06aac88b989fd: Deleting consensus metadata
I20260923 22:08:48.752787 12378 ts_tablet_manager.cc:1939] T f645844a9a404035bb0962b989a9a33a P edfa6ce1d0c94b099858a79c02a2847e: Deleting consensus metadata
I20260923 22:08:48.752907 11866 catalog_manager.cc:4841] TS e3b2cf8cbf734b48a2e06aac88b989fd (127.1.161.131:34527): tablet f645844a9a404035bb0962b989a9a33a (table test-table [id=2a70b84d1ebb413388a5ff30da154165]) successfully deleted
I20260923 22:08:48.753122 11864 catalog_manager.cc:4841] TS edfa6ce1d0c94b099858a79c02a2847e (127.1.161.130:43155): tablet f645844a9a404035bb0962b989a9a33a (table test-table [id=2a70b84d1ebb413388a5ff30da154165]) successfully deleted
W20260923 22:08:48.755949 11864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:46091 (error 108)
I20260923 22:08:48.756142 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 12076
I20260923 22:08:48.756644 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:08:48.756229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46091 (local address 127.0.0.1:58134)
0923 22:08:48.756577 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0923 22:08:48.756610 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331}
W20260923 22:08:48.756757 11864 catalog_manager.cc:4584] TS 5677e25094284de5a2f38cebe0ce5124 (127.1.161.129:46091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f645844a9a404035bb0962b989a9a33a: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46091: connect: Connection refused (error 111)
I20260923 22:08:48.761126 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 12210
I20260923 22:08:48.764955 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 11848
2026-09-23T22:08:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1695 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[12382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[12382](info): set up 2 sockets
Sep 23 22:08:48 dist-test-slave-npvd krb5kdc[12382](info): commencing operation
krb5kdc: starting...
W20260923 22:08:49.667383 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.883s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:08:49 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:08:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangeradmin_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangeradmin_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangerlookup_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangerlookup_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
I20260923 22:08:49.799588 1670 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9swwUu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/postgres -l logfile start
2026-09-23 22:08:51.945 UTC [12412] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:08:51.945 UTC [12412] LOG: listening on IPv4 address "127.1.161.148", port 47485
2026-09-23 22:08:51.948 UTC [12412] LOG: listening on Unix socket "/tmp/.s.PGSQL.47485"
2026-09-23 22:08:51.978 UTC [12415] LOG: database system was shut down at 2026-09-23 22:08:50 UTC
2026-09-23 22:08:51.981 UTC [12412] LOG: database system is ready to accept connections
I20260923 22:08:52.827642 1670 mini_postgres.cc:96] Postgres bound to 47485
2026-09-23 22:08:52.833 UTC [12423] FATAL: database "slave" does not exist
127.1.161.148:47485 - accepting connections
I20260923 22:08:52.833925 1670 mini_ranger.cc:161] Starting Ranger...
I20260923 22:08:52.846814 1670 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:08:53.263808 1670 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:08:53.263871 1670 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin
2026-09-23 22:08:53,480 [I] DB FLAVOR :POSTGRES
2026-09-23 22:08:53,480 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:08:53,481 [I] Checking connection..
2026-09-23 22:08:53,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:08:53,677 [I] Checking connection passed.
2026-09-23 22:08:53,677 [I] --------- Verifying version history table ---------
2026-09-23 22:08:53,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:53,902 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:08:53,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:54,110 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:08:54,110 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:08:54,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:08:54.324 UTC [12487] WARNING: there is no transaction in progress
2026-09-23 22:08:54,329 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:08:54,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:08:54,551 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:08:54,551 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:08:54,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:08:54,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:08:54,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-npvd.c.gcp-upstream.internal','N') ;"
2026-09-23 22:08:55,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:08:55,401 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:08:55,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:08:55,625 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:08:55,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:08:55,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:08:56,052 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:08:56,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:08:56.862 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.876 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.890 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.903 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.917 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.932 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.978 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.983 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:56.996 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.004 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.012 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.023 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.031 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.036 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.041 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57.047 UTC [12595] WARNING: there is no transaction in progress
2026-09-23 22:08:57,591 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:08:57,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:08:57,786 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:08:57,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:08:57,985 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:08:57,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:08:58,178 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:08:58,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:08:58,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:08:58,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-npvd.c.gcp-upstream.internal';"
2026-09-23 22:08:58,796 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:08:58,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:08:59,014 [I] DB_PATCHES have already been applied
I20260923 22:08:59.019302 1670 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9swwUu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:08:59.019353 1670 mini_ranger.cc:193] Using host: 127.1.161.148
I20260923 22:08:59.021907 1670 mini_ranger.cc:227] Ranger admin URL: http://127.1.161.148:53633
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:08:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:09:00.005035 1670 mini_ranger.cc:160] Time spent starting Ranger: real 7.171s user 0.000s sys 0.008s
Sep 23, 2026 10:09:17 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:09:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:09:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:09:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:09:18 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:09:18.980603 1670 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangerkms_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/rangerkms_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.161.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.161.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:09:19 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:09:19.059335 1670 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:09:19.067606 1670 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:09:19.835542 1670 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:09:19.835645 1670 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms
I20260923 22:09:19.835666 1670 mini_ranger_kms.cc:227] Using postgres at 127.1.161.148:47485
2026-09-23 22:09:19,998 [I] DB FLAVOR :POSTGRES
2026-09-23 22:09:19,999 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:09:19,999 [I] Checking connection
2026-09-23 22:09:19,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:09:20,197 [I] connection success
2026-09-23 22:09:20,198 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:09:20,198 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:09:20,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:09:20,407 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:09:20,407 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:09:20,408 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:09:20,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:47485/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:09:20,653 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:09:21.036424 1670 mini_ranger_kms.cc:310] Created kms service
I20260923 22:09:21.195089 1670 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:09:21.231038 1670 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:09:21 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201359, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:21.572300 1670 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:09:21.572606 1670 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms:/tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:09:21.572650 1670 mini_ranger_kms.cc:240] Using host: 127.1.161.148
I20260923 22:09:21.575893 1670 mini_ranger_kms.cc:276] Ranger KMS PID: 12775
I20260923 22:09:21.576025 1670 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.161.148:46761
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:09:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 23 22:09:23 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:23.651608 1670 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.592s user 0.005s sys 0.143s
I20260923 22:09:23.651711 1670 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:09:23.651793 1670 mini_ranger_kms.cc:401] 127.1.161.148:46761/kms/v1/keys
Sep 23 22:09:23 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
Sep 23 22:09:24 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
Sep 23 22:09:26 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:09:26.266223 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:37125
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:37125
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:46761/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:26.337518 12825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:26.337702 12825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:26.337740 12825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:26.338809 12825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:09:26.338861 12825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:26.338876 12825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:26.338888 12825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:09:26.338913 12825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:09:26.340371 12825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.161.148:46761/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:37125
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:09:26.340531 12825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:26.340672 12825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:26.345202 12830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:26.345242 12833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:26.345242 12831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:26.345300 12825 server_base.cc:1034] running on GCE node
I20260923 22:09:26.345618 12825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:26.345824 12825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:26.346974 12825 hybrid_clock.cc:648] HybridClock initialized: now 1790201366346969 us; error 33 us; skew 500 ppm
Sep 23 22:09:26 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:26.348925 12825 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:09:26.349254 12825 webserver.cc:460] Webserver started at http://127.1.161.190:33755/ using document root <none> and password file <none>
I20260923 22:09:26.349431 12825 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:26.349490 12825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:26.349639 12825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:26 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:26.397701 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "50e37a71cf15401e90e89bbb74d863df"
format_stamp: "Formatted at 2026-09-23 22:09:26 on dist-test-slave-npvd"
server_key: "be4b20aed7744975d108c013a3c680a2"
server_key_iv: "88545fe9d0c82b5933d9a7d124cbd49b"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:26.398088 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "50e37a71cf15401e90e89bbb74d863df"
format_stamp: "Formatted at 2026-09-23 22:09:26 on dist-test-slave-npvd"
server_key: "be4b20aed7744975d108c013a3c680a2"
server_key_iv: "88545fe9d0c82b5933d9a7d124cbd49b"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:26.399425 12825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:09:26.411692 12840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:26.411885 12825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:09:26.411974 12825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "50e37a71cf15401e90e89bbb74d863df"
format_stamp: "Formatted at 2026-09-23 22:09:26 on dist-test-slave-npvd"
server_key: "be4b20aed7744975d108c013a3c680a2"
server_key_iv: "88545fe9d0c82b5933d9a7d124cbd49b"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:26.412045 12825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:26.422667 12825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:26.424150 12825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:26.424293 12825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:26.428635 12825 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:37125
I20260923 22:09:26.428682 12892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:37125 every 8 connection(s)
I20260923 22:09:26.428969 12825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:09:26.429538 12893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:26.431317 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 12825
I20260923 22:09:26.431414 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:09:26.431471 12893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: Bootstrap starting.
I20260923 22:09:26.431587 1670 external_mini_cluster.cc:1428] Setting key 94610a84fd5e635ffb22ea3989ecaa88
I20260923 22:09:26.432060 12893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:26.432354 12893 log.cc:824] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:26.433043 12893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: No bootstrap required, opened a new log
I20260923 22:09:26.435155 12893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } }
I20260923 22:09:26.435343 12893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:26.435384 12893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e37a71cf15401e90e89bbb74d863df, State: Initialized, Role: FOLLOWER
I20260923 22:09:26.435515 12893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } }
I20260923 22:09:26.435590 12893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:09:26.435616 12893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:09:26.435659 12893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:26.436313 12893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } }
I20260923 22:09:26.436405 12893 leader_election.cc:302] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50e37a71cf15401e90e89bbb74d863df; no voters:
I20260923 22:09:26.436592 12893 leader_election.cc:288] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:09:26.436633 12898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:09:26.436784 12898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [term 1 LEADER]: Becoming Leader. State: Replica: 50e37a71cf15401e90e89bbb74d863df, State: Running, Role: LEADER
I20260923 22:09:26.436852 12893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:09:26.436918 12898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } }
I20260923 22:09:26.437273 12898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e37a71cf15401e90e89bbb74d863df. Latest consensus state: current_term: 1 leader_uuid: "50e37a71cf15401e90e89bbb74d863df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } } }
Sep 23 22:09:26 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:09:26.437408 12900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50e37a71cf15401e90e89bbb74d863df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e37a71cf15401e90e89bbb74d863df" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37125 } } }
I20260923 22:09:26.438197 12893 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:09:26.438222 12900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [sys.catalog]: This master's current role is: LEADER
I20260923 22:09:26.437470 12898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df [sys.catalog]: This master's current role is: LEADER
I20260923 22:09:26.438480 12913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:09:26.441602 12913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:09:26.444197 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:26.432010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.1.161.190:37125)
0923 22:09:26.432149 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:26.432157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:26.432186 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:09:26.432598 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:26.432606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:26.432658 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:26.432789 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:26.433430 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:26.434266 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:26.435407 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:26.436011 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:26.436130 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:26.440430 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:26.440478 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:26.440479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:26.440608 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:26.442799 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:26.443182 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:26.443185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:26.443185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:26.443277 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:26.443467 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:26.443467 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:26.443468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:26.443681 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:26.443772 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:26.443890 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 22:09:26.443980 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260923 22:09:26.447443 12913 catalog_manager.cc:1347] Generated new cluster ID: 88bc742a41424c5eb6ecb82262a5bd05
I20260923 22:09:26.447516 12913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:09:26.473224 12913 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:09:26.473860 12913 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:09:26.484259 12913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: Generated new TSK 0
I20260923 22:09:26.484436 12913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:09:27.086946 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:46761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:27.176781 12939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:27.176939 12939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:27.176959 12939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:27.178493 12939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:27.178545 12939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:27.178579 12939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:09:27.180475 12939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.161.148:46761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:09:27.180671 12939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:27.180851 12939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:27.185331 12944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:27.185341 12945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:27.185508 12947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:27.185585 12939 server_base.cc:1034] running on GCE node
I20260923 22:09:27.185802 12939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:27.186053 12939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:27.187234 12939 hybrid_clock.cc:648] HybridClock initialized: now 1790201367187214 us; error 37 us; skew 500 ppm
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:27.189941 12939 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:09:27.190354 12939 webserver.cc:460] Webserver started at http://127.1.161.129:37567/ using document root <none> and password file <none>
I20260923 22:09:27.190563 12939 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:27.190609 12939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:27.190709 12939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:27.203253 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "cde880a1ca454ba9a5809a3c0d707137"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "a5427ef67c1de2f04f44661e4bcba2be"
server_key_iv: "089e54eba43a2f1e7b589f6d3e460701"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.203631 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "cde880a1ca454ba9a5809a3c0d707137"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "a5427ef67c1de2f04f44661e4bcba2be"
server_key_iv: "089e54eba43a2f1e7b589f6d3e460701"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.211495 12939 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260923 22:09:27.221745 12956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.221921 12939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:09:27.222002 12939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "cde880a1ca454ba9a5809a3c0d707137"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "a5427ef67c1de2f04f44661e4bcba2be"
server_key_iv: "089e54eba43a2f1e7b589f6d3e460701"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.222071 12939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:27.246515 12939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:27.248382 12939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:27.248524 12939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:27.248728 12939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:27.249424 12939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:27.249466 12939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.249549 12939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:27.249588 12939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.255518 12939 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:45819
I20260923 22:09:27.255945 12939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:09:27.262251 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 12939
I20260923 22:09:27.262346 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:09:27.262499 13069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:45819 every 8 connection(s)
I20260923 22:09:27.262535 1670 external_mini_cluster.cc:1428] Setting key 8f6854dc5637c8da656e4c3461e18894
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:09:27.277626 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.256613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:34335 (local address 127.1.161.190:37125)
0923 22:09:27.256737 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.256739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.259444 (+ 2705us) server_negotiation.cc:407] Connection header received
0923 22:09:27.260077 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.260078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.260093 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.260133 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:27.260848 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.261540 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.263126 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.263434 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.263489 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.273177 (+ 9688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:27.273178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:27.273179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:27.273185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:27.274784 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.276630 (+ 1846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.276631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.276632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.276689 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.276962 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.276963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.276963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.277084 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:27.277130 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.277452 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.277530 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260923 22:09:27.278163 13072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.256577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37125 (local address 127.1.161.129:34335)
0923 22:09:27.259325 (+ 2748us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.259366 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.259958 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.260298 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.260302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.260522 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:27.260750 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.260756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.261642 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.261643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.263034 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.263037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.263570 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.263571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.263698 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.271623 (+ 7925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:27.271631 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:27.273075 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:27.274868 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.274873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.274876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.275108 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.276755 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.276755 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.276755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.276886 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.277158 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:27.277163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:27.277298 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.277798 (+ 500us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.277937 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2708,"thread_start_us":66,"threads_started":1}
I20260923 22:09:27.278738 13070 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37125
I20260923 22:09:27.278828 13070 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:27.278998 13070 heartbeater.cc:511] Master 127.1.161.190:37125 requested a full tablet report, sending...
I20260923 22:09:27.279506 12857 ts_manager.cc:194] Registered new tserver with Master: cde880a1ca454ba9a5809a3c0d707137 (127.1.161.129:45819)
I20260923 22:09:27.280009 12857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:34335
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:09:27.299274 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:46761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:09:27.411339 13078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:27.411505 13078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:27.411525 13078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:27.412611 13078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:27.412652 13078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:27.412675 13078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:09:27.413981 13078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.161.148:46761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:09:27.414160 13078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:27.414403 13078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:27.419327 13086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:27.419595 13078 server_base.cc:1034] running on GCE node
W20260923 22:09:27.419708 13083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:27.419600 13084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:27.420013 13078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:27.420240 13078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:27.421396 13078 hybrid_clock.cc:648] HybridClock initialized: now 1790201367421377 us; error 33 us; skew 500 ppm
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:27.424039 13078 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:09:27.424455 13078 webserver.cc:460] Webserver started at http://127.1.161.130:37631/ using document root <none> and password file <none>
I20260923 22:09:27.424650 13078 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:27.424710 13078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:27.424835 13078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:27.436338 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "da7faddae87c49dea6e577173d3e4b0d"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "f9f80f51a296cb104fd766e5da5faf8f"
server_key_iv: "3c88d3b52e0872d720d2d72c4aaa452a"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.436808 13078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "da7faddae87c49dea6e577173d3e4b0d"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "f9f80f51a296cb104fd766e5da5faf8f"
server_key_iv: "3c88d3b52e0872d720d2d72c4aaa452a"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.438460 13078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:09:27.460194 13094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.460359 13078 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260923 22:09:27.460424 13078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "da7faddae87c49dea6e577173d3e4b0d"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "f9f80f51a296cb104fd766e5da5faf8f"
server_key_iv: "3c88d3b52e0872d720d2d72c4aaa452a"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.460484 13078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:27.479332 13078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:27.481865 13078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:27.482034 13078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:27.482235 13078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:27.482545 13078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:27.482612 13078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.482662 13078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:27.482702 13078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.489884 13078 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:38319
I20260923 22:09:27.490211 13078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:09:27.495424 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:38319 every 8 connection(s)
I20260923 22:09:27.498771 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 13078
I20260923 22:09:27.498869 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:09:27.499061 1670 external_mini_cluster.cc:1428] Setting key d3d2257b88bce13a65fd4ccff07585a5
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:09:27.507855 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.496385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:37431 (local address 127.1.161.190:37125)
0923 22:09:27.496441 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.496443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.496453 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:09:27.497019 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.497020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.497032 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.497060 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:27.497786 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.498495 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.501999 (+ 3504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.502301 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.502345 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.504703 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:27.504704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:27.504704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:27.504713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:27.506378 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.506805 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.506806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.506807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.506854 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.507103 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.507103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.507104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.507273 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:27.507316 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.507659 (+ 343us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.507739 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:09:27.508502 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.496079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37125 (local address 127.1.161.130:37431)
0923 22:09:27.496259 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.496319 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.496877 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.497236 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.497240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.497437 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:27.497688 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.497693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.498605 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.498606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.501884 (+ 3278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.501887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.502433 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.502433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.502559 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.503063 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:27.503071 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:27.504605 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:27.506481 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.506486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.506488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.506727 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.506926 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.506927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.506927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.507036 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.507361 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:27.507366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:27.507523 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.508136 (+ 613us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.508277 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260923 22:09:27.509066 13208 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37125
I20260923 22:09:27.509150 13208 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:27.509301 13208 heartbeater.cc:511] Master 127.1.161.190:37125 requested a full tablet report, sending...
I20260923 22:09:27.509670 12857 ts_manager.cc:194] Registered new tserver with Master: da7faddae87c49dea6e577173d3e4b0d (127.1.161.130:38319)
I20260923 22:09:27.510097 12857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:37431
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:09:27.547825 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:46761/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:09:27.671126 13213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:09:27.671479 13213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:09:27.671564 13213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:09:27.672983 13213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:09:27.673084 13213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:09:27.673161 13213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:09:27.675349 13213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.161.148:46761/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:09:27.675827 13213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:09:27.676147 13213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:09:27.682472 13219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:09:27.682838 13218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:27.683602 13213 server_base.cc:1034] running on GCE node
W20260923 22:09:27.687306 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:09:27.687595 13213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:09:27.687846 13213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:09:27.689033 13213 hybrid_clock.cc:648] HybridClock initialized: now 1790201367689012 us; error 36 us; skew 500 ppm
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:09:27.692092 13213 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:09:27.692564 13213 webserver.cc:460] Webserver started at http://127.1.161.131:35893/ using document root <none> and password file <none>
I20260923 22:09:27.692811 13213 fs_manager.cc:359] Metadata directory not provided
I20260923 22:09:27.692858 13213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:09:27.692971 13213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:27.706828 13213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "b66fd3d1a75641aa8649f3553968f2b7"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "21c457bd3bb9f3f2455d3c3ac7357040"
server_key_iv: "5b0947bbd50ec2b9892bbc7e1ae45dc2"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.707341 13213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "b66fd3d1a75641aa8649f3553968f2b7"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "21c457bd3bb9f3f2455d3c3ac7357040"
server_key_iv: "5b0947bbd50ec2b9892bbc7e1ae45dc2"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.716701 13213 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.001s
I20260923 22:09:27.734587 13228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.734807 13213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:09:27.734913 13213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b66fd3d1a75641aa8649f3553968f2b7"
format_stamp: "Formatted at 2026-09-23 22:09:27 on dist-test-slave-npvd"
server_key: "21c457bd3bb9f3f2455d3c3ac7357040"
server_key_iv: "5b0947bbd50ec2b9892bbc7e1ae45dc2"
server_key_version: "kuduclusterkey@0"
I20260923 22:09:27.734987 13213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:09:27.744333 13213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:09:27.746651 13213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:09:27.746834 13213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:09:27.747077 13213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:09:27.747545 13213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:09:27.747638 13213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.747717 13213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:09:27.747778 13213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:09:27.753813 13213 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:37939
I20260923 22:09:27.754223 13213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:09:27.757439 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 13213
I20260923 22:09:27.757531 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:09:27.757766 1670 external_mini_cluster.cc:1428] Setting key 0bee7d971193d9d86f771610ed1f5a6a
I20260923 22:09:27.763710 13341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:37939 every 8 connection(s)
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:09:27.778596 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.765064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:38027 (local address 127.1.161.190:37125)
0923 22:09:27.765109 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.765112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.766534 (+ 1422us) server_negotiation.cc:407] Connection header received
0923 22:09:27.768558 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.768559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.768569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.768595 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:27.769298 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.770075 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.771518 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.771797 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.771834 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.774112 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:27.774113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:27.774113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:27.774120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:27.775801 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.777546 (+ 1745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.777547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.777548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.777604 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.777925 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.777925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.777926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.778057 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:27.778097 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.778419 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.778492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:09:27.779412 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.764806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37125 (local address 127.1.161.131:38027)
0923 22:09:27.766397 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.766444 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.768445 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.768753 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.768757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.768958 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:09:27.769203 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.769209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.770188 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.770189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.771423 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.771426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.771910 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.771911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.772034 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.772537 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:09:27.772546 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:09:27.774007 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:09:27.776501 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.776507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.776509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.776761 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.777711 (+ 950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:09:27.777712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:09:27.777712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:09:27.777853 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:09:27.778140 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:09:27.778146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:09:27.778284 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.778907 (+ 623us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.779061 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"thread_start_us":77,"threads_started":1}
I20260923 22:09:27.779975 13342 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37125
I20260923 22:09:27.780067 13342 heartbeater.cc:464] Registering TS with master...
I20260923 22:09:27.780224 13342 heartbeater.cc:511] Master 127.1.161.190:37125 requested a full tablet report, sending...
I20260923 22:09:27.780580 12857 ts_manager.cc:194] Registered new tserver with Master: b66fd3d1a75641aa8649f3553968f2b7 (127.1.161.131:37939)
I20260923 22:09:27.780933 12857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:38027
I20260923 22:09:27.790318 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:09:27.800662 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.793003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.1.161.190:37125)
0923 22:09:27.793036 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.793038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.793351 (+ 313us) server_negotiation.cc:407] Connection header received
0923 22:09:27.793430 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.793430 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.793439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.793462 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:09:27.794113 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.794674 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.795830 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.796149 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.796200 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.798155 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:09:27.798157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:09:27.798157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:09:27.798163 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:09:27.799618 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.799937 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.799938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.799938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.799983 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:09:27.800162 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:09:27.800163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:09:27.800163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:09:27.800259 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:09:27.800321 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.800518 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.800583 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:09:27.807044 12857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:09:27.809239 12857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 23 22:09:27 dist-test-slave-npvd krb5kdc[12382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:09:27.861789 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.841059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:37939 (local address 127.0.0.1:44078)
0923 22:09:27.841539 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.841546 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.841578 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.842057 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.842060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.842063 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:27.842143 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.842145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.859429 (+ 17284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.859430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.860606 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.860609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.861500 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.861501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.861586 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.861604 (+ 18us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.861654 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.861701 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":48,"threads_started":1}
I20260923 22:09:27.862397 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.841282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.1.161.131:37939)
0923 22:09:27.841719 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.841724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.841734 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:09:27.841789 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.841791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.841806 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.854610 (+ 12804us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:27.854695 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.859353 (+ 4658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.860717 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.861376 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.862161 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.862264 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.862289 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.862324 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":70,"threads_started":1}
I20260923 22:09:27.863175 13265 tablet_service.cc:1467] Processing CreateTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea (DEFAULT_TABLE table=test-table [id=e6eb443acd8046cd8b239b0bbcaa4ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:27.863538 13265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3579e0b38d7a4e88a23da5f3c66a10ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:27.867084 13361 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Bootstrap starting.
I20260923 22:09:27.868019 13361 tablet_bootstrap.cc:654] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:27.868848 13361 log.cc:824] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:27.869541 13361 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: No bootstrap required, opened a new log
I20260923 22:09:27.869591 13361 ts_tablet_manager.cc:1397] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:09:27.871368 13361 raft_consensus.cc:348] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.871515 13361 raft_consensus.cc:374] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:27.871546 13361 raft_consensus.cc:729] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66fd3d1a75641aa8649f3553968f2b7, State: Initialized, Role: FOLLOWER
I20260923 22:09:27.871640 13361 consensus_queue.cc:260] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.871855 13361 ts_tablet_manager.cc:1428] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:09:27.872020 13342 heartbeater.cc:503] Master 127.1.161.190:37125 was elected leader, sending a full tablet report...
I20260923 22:09:27.872216 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.840840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38319 (local address 127.0.0.1:46804)
0923 22:09:27.841338 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.841375 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.841438 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.866859 (+ 25421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.866860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.866862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:27.866903 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.866904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.868244 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.868246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.869669 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.869672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.872034 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.872035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.872088 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.872091 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.872119 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.872150 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":61,"threads_started":1}
I20260923 22:09:27.872505 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.841070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.1.161.130:38319)
0923 22:09:27.866707 (+ 25637us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.866711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.866722 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:09:27.866757 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.866759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.866774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.866965 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:27.867072 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.868051 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.869781 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.870458 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.872294 (+ 1836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.872372 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.872398 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.872437 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25597,"thread_start_us":69,"threads_started":1}
I20260923 22:09:27.873180 13142 tablet_service.cc:1467] Processing CreateTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea (DEFAULT_TABLE table=test-table [id=e6eb443acd8046cd8b239b0bbcaa4ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:27.873435 13142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3579e0b38d7a4e88a23da5f3c66a10ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:27.877689 13364 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Bootstrap starting.
I20260923 22:09:27.878639 13364 tablet_bootstrap.cc:654] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:27.878958 13364 log.cc:824] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:27.879642 13364 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: No bootstrap required, opened a new log
I20260923 22:09:27.879698 13364 ts_tablet_manager.cc:1397] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:09:27.881381 13364 raft_consensus.cc:348] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.881544 13364 raft_consensus.cc:374] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:27.881580 13364 raft_consensus.cc:729] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7faddae87c49dea6e577173d3e4b0d, State: Initialized, Role: FOLLOWER
I20260923 22:09:27.881686 13364 consensus_queue.cc:260] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.881916 13364 ts_tablet_manager.cc:1428] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:09:27.882082 13208 heartbeater.cc:503] Master 127.1.161.190:37125 was elected leader, sending a full tablet report...
I20260923 22:09:27.883455 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.841198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:45819 (local address 127.0.0.1:36804)
0923 22:09:27.841905 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:27.841913 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:27.841948 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:27.870807 (+ 28859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:27.870808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:27.870810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:27.870848 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.870850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.872602 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.872602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.873490 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:27.873493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.883271 (+ 9778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.883272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:27.883326 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.883331 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:09:27.883360 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:09:27.883393 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":52,"threads_started":1}
I20260923 22:09:27.883935 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:27.841283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.1.161.129:45819)
0923 22:09:27.870613 (+ 29330us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:27.870617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:27.870627 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:09:27.870658 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:27.870660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:27.870675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:27.870909 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:27.870967 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.871919 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.873582 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:27.874330 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:27.883742 (+ 9412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:27.883808 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:27.883831 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 22:09:27.883870 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29293,"thread_start_us":57,"threads_started":1}
I20260923 22:09:27.884560 12998 tablet_service.cc:1467] Processing CreateTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea (DEFAULT_TABLE table=test-table [id=e6eb443acd8046cd8b239b0bbcaa4ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:09:27.884841 12998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3579e0b38d7a4e88a23da5f3c66a10ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:09:27.888549 13367 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Bootstrap starting.
I20260923 22:09:27.889202 13367 tablet_bootstrap.cc:654] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Neither blocks nor log segments found. Creating new log.
I20260923 22:09:27.889555 13367 log.cc:824] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Log is configured to *not* fsync() on all Append() calls
I20260923 22:09:27.901242 13367 tablet_bootstrap.cc:492] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: No bootstrap required, opened a new log
I20260923 22:09:27.901304 13367 ts_tablet_manager.cc:1397] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260923 22:09:27.905962 13367 raft_consensus.cc:348] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.906203 13367 raft_consensus.cc:374] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:09:27.906279 13367 raft_consensus.cc:729] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde880a1ca454ba9a5809a3c0d707137, State: Initialized, Role: FOLLOWER
I20260923 22:09:27.906483 13367 consensus_queue.cc:260] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:27.906756 13367 ts_tablet_manager.cc:1428] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260923 22:09:27.907466 13070 heartbeater.cc:503] Master 127.1.161.190:37125 was elected leader, sending a full tablet report...
W20260923 22:09:28.008169 13209 tablet.cc:2369] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:09:28.011500 13366 raft_consensus.cc:482] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:09:28.011608 13366 raft_consensus.cc:504] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:28.011938 13366 leader_election.cc:288] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cde880a1ca454ba9a5809a3c0d707137 (127.1.161.129:45819), b66fd3d1a75641aa8649f3553968f2b7 (127.1.161.131:37939)
W20260923 22:09:28.014446 13071 tablet.cc:2369] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:09:28.015610 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.012261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:37223 (local address 127.1.161.129:45819)
0923 22:09:28.012326 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:28.012328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:28.012428 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:09:28.012455 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:28.012456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:28.012467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:28.012517 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:28.012688 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.013353 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.014681 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.015224 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.015286 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:28.015294 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:28.015506 (+ 212us) server_negotiation.cc:299] Negotiation successful
0923 22:09:28.015544 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:09:28.015811 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.012101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:45819 (local address 127.1.161.130:37223)
0923 22:09:28.012315 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:28.012325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:28.012377 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:28.012562 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:28.012563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:28.012565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:28.012626 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:28.012628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.013459 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.013459 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:28.014587 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:28.014589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.015348 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.015349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:28.015430 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:28.015433 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:09:28.015690 (+ 257us) client_negotiation.cc:240] Negotiation successful
0923 22:09:28.015732 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":48,"threads_started":1}
I20260923 22:09:28.015877 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.012362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:50881 (local address 127.1.161.131:37939)
0923 22:09:28.012414 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:28.012416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:28.012484 (+ 68us) server_negotiation.cc:407] Connection header received
0923 22:09:28.012567 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:28.012568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:28.012576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:28.012601 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:09:28.012777 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.013483 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.014790 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.015396 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.015443 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:28.015450 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:28.015768 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 22:09:28.015803 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:09:28.016206 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.012171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:37939 (local address 127.1.161.130:50881)
0923 22:09:28.012436 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:28.012443 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:09:28.012504 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:09:28.012656 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:09:28.012657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:09:28.012658 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:09:28.012717 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:28.012718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.013587 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.013588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:28.014697 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:09:28.014700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.015555 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.015557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:09:28.015639 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:28.015642 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:09:28.015974 (+ 332us) client_negotiation.cc:240] Negotiation successful
0923 22:09:28.016119 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":63,"threads_started":1}
I20260923 22:09:28.016263 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3579e0b38d7a4e88a23da5f3c66a10ea" candidate_uuid: "da7faddae87c49dea6e577173d3e4b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cde880a1ca454ba9a5809a3c0d707137" is_pre_election: true
I20260923 22:09:28.016428 13024 raft_consensus.cc:2454] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7faddae87c49dea6e577173d3e4b0d in term 0.
I20260923 22:09:28.016486 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3579e0b38d7a4e88a23da5f3c66a10ea" candidate_uuid: "da7faddae87c49dea6e577173d3e4b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66fd3d1a75641aa8649f3553968f2b7" is_pre_election: true
I20260923 22:09:28.016628 13277 raft_consensus.cc:2454] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7faddae87c49dea6e577173d3e4b0d in term 0.
I20260923 22:09:28.016645 13097 leader_election.cc:302] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cde880a1ca454ba9a5809a3c0d707137, da7faddae87c49dea6e577173d3e4b0d; no voters:
I20260923 22:09:28.016780 13366 raft_consensus.cc:2781] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:09:28.016810 13366 raft_consensus.cc:482] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:09:28.016827 13366 raft_consensus.cc:3037] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:28.017575 13366 raft_consensus.cc:504] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:28.017696 13366 leader_election.cc:288] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [CANDIDATE]: Term 1 election: Requested vote from peers cde880a1ca454ba9a5809a3c0d707137 (127.1.161.129:45819), b66fd3d1a75641aa8649f3553968f2b7 (127.1.161.131:37939)
I20260923 22:09:28.017817 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3579e0b38d7a4e88a23da5f3c66a10ea" candidate_uuid: "da7faddae87c49dea6e577173d3e4b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66fd3d1a75641aa8649f3553968f2b7"
I20260923 22:09:28.017850 13024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3579e0b38d7a4e88a23da5f3c66a10ea" candidate_uuid: "da7faddae87c49dea6e577173d3e4b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cde880a1ca454ba9a5809a3c0d707137"
I20260923 22:09:28.017881 13277 raft_consensus.cc:3037] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:09:28.017920 13024 raft_consensus.cc:3037] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 0 FOLLOWER]: Advancing to term 1
W20260923 22:09:28.018038 13343 tablet.cc:2369] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:09:28.018647 13024 raft_consensus.cc:2454] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7faddae87c49dea6e577173d3e4b0d in term 1.
I20260923 22:09:28.018673 13277 raft_consensus.cc:2454] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7faddae87c49dea6e577173d3e4b0d in term 1.
I20260923 22:09:28.018811 13097 leader_election.cc:302] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cde880a1ca454ba9a5809a3c0d707137, da7faddae87c49dea6e577173d3e4b0d; no voters:
I20260923 22:09:28.018877 13366 raft_consensus.cc:2781] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:09:28.018985 13366 raft_consensus.cc:686] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 1 LEADER]: Becoming Leader. State: Replica: da7faddae87c49dea6e577173d3e4b0d, State: Running, Role: LEADER
I20260923 22:09:28.019058 13366 consensus_queue.cc:237] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } }
I20260923 22:09:28.019790 12857 catalog_manager.cc:5495] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d reported cstate change: term changed from 0 to 1, leader changed from <none> to da7faddae87c49dea6e577173d3e4b0d (127.1.161.130). New cstate: current_term: 1 leader_uuid: "da7faddae87c49dea6e577173d3e4b0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7faddae87c49dea6e577173d3e4b0d" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 } health_report { overall_health: UNKNOWN } } }
I20260923 22:09:28.034382 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.031518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.1.161.130:38319)
0923 22:09:28.031553 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:09:28.031555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:09:28.031597 (+ 42us) server_negotiation.cc:407] Connection header received
0923 22:09:28.031658 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:09:28.031659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:09:28.031675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:09:28.031751 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:09:28.031876 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.032546 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.033542 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:09:28.033838 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:09:28.033905 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:09:28.034051 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:09:28.034175 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:09:28.034260 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 22:09:28.034288 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 22:09:28.034307 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:09:28.036064 13024 raft_consensus.cc:1261] T 3579e0b38d7a4e88a23da5f3c66a10ea P cde880a1ca454ba9a5809a3c0d707137 [term 1 FOLLOWER]: Refusing update from remote peer da7faddae87c49dea6e577173d3e4b0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:09:28.036067 13277 raft_consensus.cc:1261] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 1 FOLLOWER]: Refusing update from remote peer da7faddae87c49dea6e577173d3e4b0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:09:28.036343 13366 consensus_queue.cc:1035] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66fd3d1a75641aa8649f3553968f2b7" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:09:28.036521 13366 consensus_queue.cc:1035] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cde880a1ca454ba9a5809a3c0d707137" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:09:28.039492 13375 mvcc.cc:205] Tried to move back new op lower bound from 7332664803473424384 to 7332664803406659584. Current Snapshot: MvccSnapshot[applied={T|T < 7332664803473424384}]
I20260923 22:09:28.040442 13376 mvcc.cc:205] Tried to move back new op lower bound from 7332664803473424384 to 7332664803406659584. Current Snapshot: MvccSnapshot[applied={T|T < 7332664803473424384}]
I20260923 22:09:28.042493 12856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:09:28.042570 12856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:09:28.043275 12856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50e37a71cf15401e90e89bbb74d863df: Sending DeleteTablet for 3 replicas of tablet 3579e0b38d7a4e88a23da5f3c66a10ea
I20260923 22:09:28.043694 13142 tablet_service.cc:1514] Processing DeleteTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:28 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:46804
I20260923 22:09:28.043860 13265 tablet_service.cc:1514] Processing DeleteTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:28 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:44078
I20260923 22:09:28.044034 12998 tablet_service.cc:1514] Processing DeleteTablet for tablet 3579e0b38d7a4e88a23da5f3c66a10ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:09:28 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:36804
I20260923 22:09:28.044570 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 12939
I20260923 22:09:28.047290 13383 tablet_replica.cc:330] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: stopping tablet replica
I20260923 22:09:28.047395 13384 tablet_replica.cc:330] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: stopping tablet replica
I20260923 22:09:28.050868 13384 raft_consensus.cc:2229] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 22:09:28.050771 13097 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:45819 (error 108)
I20260923 22:09:28.051224 13383 raft_consensus.cc:2229] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:09:28.051411 13384 raft_consensus.cc:2258] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:09:28.051591 13383 raft_consensus.cc:2258] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:09:28.052055 13384 ts_tablet_manager.cc:1905] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:09:28.052412 13383 ts_tablet_manager.cc:1905] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:09:28.055282 13383 ts_tablet_manager.cc:1918] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:09:28.055482 13384 ts_tablet_manager.cc:1918] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:09:28.055593 13383 log.cc:1196] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/3579e0b38d7a4e88a23da5f3c66a10ea
I20260923 22:09:28.055699 13384 log.cc:1196] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/3579e0b38d7a4e88a23da5f3c66a10ea
I20260923 22:09:28.056049 13383 ts_tablet_manager.cc:1939] T 3579e0b38d7a4e88a23da5f3c66a10ea P da7faddae87c49dea6e577173d3e4b0d: Deleting consensus metadata
I20260923 22:09:28.056679 13384 ts_tablet_manager.cc:1939] T 3579e0b38d7a4e88a23da5f3c66a10ea P b66fd3d1a75641aa8649f3553968f2b7: Deleting consensus metadata
I20260923 22:09:28.057089 12842 catalog_manager.cc:4841] TS da7faddae87c49dea6e577173d3e4b0d (127.1.161.130:38319): tablet 3579e0b38d7a4e88a23da5f3c66a10ea (table test-table [id=e6eb443acd8046cd8b239b0bbcaa4ce8]) successfully deleted
I20260923 22:09:28.057538 12842 catalog_manager.cc:4841] TS b66fd3d1a75641aa8649f3553968f2b7 (127.1.161.131:37939): tablet 3579e0b38d7a4e88a23da5f3c66a10ea (table test-table [id=e6eb443acd8046cd8b239b0bbcaa4ce8]) successfully deleted
W20260923 22:09:28.057760 12843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:45819 (error 108)
I20260923 22:09:28.058614 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:09:28.058516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:45819 (local address 127.0.0.1:36810)
0923 22:09:28.058550 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:09:28.058584 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:45819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:09:28.058614 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 13078
W20260923 22:09:28.059518 12843 catalog_manager.cc:4584] TS cde880a1ca454ba9a5809a3c0d707137 (127.1.161.129:45819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3579e0b38d7a4e88a23da5f3c66a10ea: Network error: Client connection negotiation failed: client connection to 127.1.161.129:45819: connect: Connection refused (error 111)
I20260923 22:09:28.065059 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 13213
I20260923 22:09:28.070453 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 12825
I20260923 22:09:28.075579 1670 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:09:28.409384 1670 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:09:28.409471 1670 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:09:28.754269 1670 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:09:28.754 UTC [12412] LOG: received smart shutdown request
2026-09-23 22:09:28.757 UTC [12412] LOG: background worker "logical replication launcher" (PID 12421) exited with exit code 1
2026-09-23 22:09:28.757 UTC [12416] LOG: shutting down
2026-09-23 22:09:28.914 UTC [12412] LOG: database system is shut down
2026-09-23T22:09:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (40243 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:09:29 dist-test-slave-npvd krb5kdc[13396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:09:29 dist-test-slave-npvd krb5kdc[13396](info): set up 2 sockets
Sep 23 22:09:29 dist-test-slave-npvd krb5kdc[13396](info): commencing operation
krb5kdc: starting...
W20260923 22:09:29.939005 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.912s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:09:30 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:09:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangeradmin_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangeradmin_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangerlookup_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangerlookup_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
I20260923 22:09:30.071635 1670 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9swwUu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/postgres -l logfile start
2026-09-23 22:09:32.249 UTC [13426] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 22:09:32.249 UTC [13426] LOG: listening on IPv4 address "127.1.161.148", port 38455
2026-09-23 22:09:32.252 UTC [13426] LOG: listening on Unix socket "/tmp/.s.PGSQL.38455"
2026-09-23 22:09:32.281 UTC [13429] LOG: database system was shut down at 2026-09-23 22:09:30 UTC
2026-09-23 22:09:32.284 UTC [13426] LOG: database system is ready to accept connections
I20260923 22:09:33.142944 1670 mini_postgres.cc:96] Postgres bound to 38455
2026-09-23 22:09:33.148 UTC [13437] FATAL: database "slave" does not exist
127.1.161.148:38455 - accepting connections
I20260923 22:09:33.149250 1670 mini_ranger.cc:161] Starting Ranger...
I20260923 22:09:33.162726 1670 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 22:09:33.589416 1670 mini_ranger.cc:87] Created ranger Postgres database
I20260923 22:09:33.589488 1670 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin
2026-09-23 22:09:33,832 [I] DB FLAVOR :POSTGRES
2026-09-23 22:09:33,832 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:09:33,832 [I] Checking connection..
2026-09-23 22:09:33,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 22:09:34,058 [I] Checking connection passed.
2026-09-23 22:09:34,058 [I] --------- Verifying version history table ---------
2026-09-23 22:09:34,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:34,285 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:09:34,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:34,528 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 22:09:34,529 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 22:09:34,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 22:09:34.743 UTC [13501] WARNING: there is no transaction in progress
2026-09-23 22:09:34,748 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 22:09:34,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 22:09:34,985 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 22:09:34,985 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:09:34,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 22:09:35,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 22:09:35,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-npvd.c.gcp-upstream.internal','N') ;"
2026-09-23 22:09:35,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:09:35,811 [I] Table x_portal_user does not exist in database ranger
2026-09-23 22:09:35,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:09:36,041 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 22:09:36,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:09:36,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:09:36,454 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 22:09:36,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 22:09:37.286 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.301 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.317 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.331 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.346 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.363 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.423 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.430 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.442 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.451 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.460 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.472 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.480 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.485 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.491 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:37.496 UTC [13609] WARNING: there is no transaction in progress
2026-09-23 22:09:38,062 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 22:09:38,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 22:09:38,281 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 22:09:38,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 22:09:38,507 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 22:09:38,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 22:09:38,726 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 22:09:38,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 22:09:38,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 22:09:39,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-npvd.c.gcp-upstream.internal';"
2026-09-23 22:09:39,359 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 22:09:39,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 22:09:39,577 [I] DB_PATCHES have already been applied
I20260923 22:09:39.582651 1670 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9swwUu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 22:09:39.582712 1670 mini_ranger.cc:193] Using host: 127.1.161.148
I20260923 22:09:39.585496 1670 mini_ranger.cc:227] Ranger admin URL: http://127.1.161.148:49965
Sep 23, 2026 10:09:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:09:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:09:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 10:09:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 10:09:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 10:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 10:09:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 22:09:40.601404 1670 mini_ranger.cc:160] Time spent starting Ranger: real 7.452s user 0.000s sys 0.008s
Sep 23, 2026 10:09:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 10:09:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 10:09:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 10:09:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 10:09:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 22:09:59.640195 1670 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.161.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.161.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangerkms_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.161.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/rangerkms_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.161.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.161.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.161.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/HTTP_127.1.161.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 22:09:59 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201399, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 22:09:59.709870 1670 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 22:09:59.717146 1670 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 22:10:00.486671 1670 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 22:10:00.486755 1670 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms
I20260923 22:10:00.486771 1670 mini_ranger_kms.cc:227] Using postgres at 127.1.161.148:38455
2026-09-23 22:10:00,633 [I] DB FLAVOR :POSTGRES
2026-09-23 22:10:00,633 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 22:10:00,633 [I] Checking connection
2026-09-23 22:10:00,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 22:10:00,827 [I] connection success
2026-09-23 22:10:00,828 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 22:10:00,828 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 22:10:00,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 22:10:01,029 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 22:10:01,029 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 22:10:01,029 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 22:10:01,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.161.148:38455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 22:10:01,244 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 22:10:01.640915 1670 mini_ranger_kms.cc:310] Created kms service
I20260923 22:10:01.823820 1670 mini_ranger_kms.cc:326] Created kudu user
I20260923 22:10:01.862149 1670 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 22:10:01 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201399, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:02.195206 1670 mini_ranger_kms.cc:384] Added ranger policy
I20260923 22:10:02.195303 1670 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms:/tmp/dist-test-task9swwUu/build/release/bin/postgresql.jar:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9swwUu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 22:10:02.195319 1670 mini_ranger_kms.cc:240] Using host: 127.1.161.148
I20260923 22:10:02.198410 1670 mini_ranger_kms.cc:276] Ranger KMS PID: 13789
I20260923 22:10:02.198480 1670 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.161.148:39587
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 10:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 22:10:04.170027 1670 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.460s user 0.000s sys 0.133s
I20260923 22:10:04.170109 1670 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 22:10:04.170193 1670 mini_ranger_kms.cc:401] 127.1.161.148:39587/kms/v1/keys
Sep 23 22:10:04 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 22:10:04 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
Sep 23 22:10:05 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201404, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.161.148@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
Sep 23 22:10:07 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:10:07.071784 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41873
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41873
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:07.159206 13840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:07.159425 13840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:07.159452 13840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:07.161034 13840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:07.161093 13840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:07.161114 13840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:07.161130 13840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:07.161145 13840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:07.162945 13840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.161.148:39587/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41873
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:07.163136 13840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:07.163360 13840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:07.167913 13846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:07.168010 13848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:07.167896 13840 server_base.cc:1034] running on GCE node
W20260923 22:10:07.167879 13845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:07.168366 13840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:07.168635 13840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:07.169776 13840 hybrid_clock.cc:648] HybridClock initialized: now 1790201407169766 us; error 24 us; skew 500 ppm
Sep 23 22:10:07 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:07.171939 13840 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:07.172271 13840 webserver.cc:460] Webserver started at http://127.1.161.190:38557/ using document root <none> and password file <none>
I20260923 22:10:07.172441 13840 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:07.172483 13840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:07.172551 13840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:07 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:07.220890 13840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "46a5897d3c2147f09eab16e638d615c3"
format_stamp: "Formatted at 2026-09-23 22:10:07 on dist-test-slave-npvd"
server_key: "bd30cbb524971b9571646f548cbaf145"
server_key_iv: "22bc6c9e595d115f0a199a1a61f5e23f"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:07.221276 13840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "46a5897d3c2147f09eab16e638d615c3"
format_stamp: "Formatted at 2026-09-23 22:10:07 on dist-test-slave-npvd"
server_key: "bd30cbb524971b9571646f548cbaf145"
server_key_iv: "22bc6c9e595d115f0a199a1a61f5e23f"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:07.222782 13840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:10:07.234575 13855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:07.234722 13840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:07.234803 13840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "46a5897d3c2147f09eab16e638d615c3"
format_stamp: "Formatted at 2026-09-23 22:10:07 on dist-test-slave-npvd"
server_key: "bd30cbb524971b9571646f548cbaf145"
server_key_iv: "22bc6c9e595d115f0a199a1a61f5e23f"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:07.234881 13840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:07.256263 13840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:07.258097 13840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:07.258234 13840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:07.262331 13840 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41873
I20260923 22:10:07.262364 13907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41873 every 8 connection(s)
I20260923 22:10:07.262650 13840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:07.263345 13908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:07.265877 13908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Bootstrap starting.
I20260923 22:10:07.266445 13908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:07.266659 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 13840
I20260923 22:10:07.266760 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:10:07.266943 1670 external_mini_cluster.cc:1428] Setting key 971ae19f0ebd31bf5b4e457ea690db6f
I20260923 22:10:07.267520 13908 log.cc:824] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:07.268380 13908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: No bootstrap required, opened a new log
I20260923 22:10:07.270164 13908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:07.270397 13908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:07.270460 13908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a5897d3c2147f09eab16e638d615c3, State: Initialized, Role: FOLLOWER
I20260923 22:10:07.270604 13908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:07.270687 13908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:07.270723 13908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:07.270819 13908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:07.271364 13908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:07.271535 13908 leader_election.cc:302] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a5897d3c2147f09eab16e638d615c3; no voters:
I20260923 22:10:07.271730 13908 leader_election.cc:288] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:07.271870 13913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:07.272255 13908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:07.272404 13913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 LEADER]: Becoming Leader. State: Replica: 46a5897d3c2147f09eab16e638d615c3, State: Running, Role: LEADER
I20260923 22:10:07.272519 13913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:07.272873 13913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a5897d3c2147f09eab16e638d615c3. Latest consensus state: current_term: 1 leader_uuid: "46a5897d3c2147f09eab16e638d615c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } } }
I20260923 22:10:07.273005 13913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:07.273442 13914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46a5897d3c2147f09eab16e638d615c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } } }
I20260923 22:10:07.273541 13914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: This master's current role is: LEADER
Sep 23 22:10:07 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:07.273943 13922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:07.273981 13908 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:10:07.276823 13922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:07.278930 13922 catalog_manager.cc:1347] Generated new cluster ID: 17f94dfa4a33470c8949039269982e6b
I20260923 22:10:07.279008 13922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:07.280390 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:07.267564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.1.161.190:41873)
0923 22:10:07.267729 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:07.267753 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:07.267903 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:10:07.268265 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:07.268271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:07.268449 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:07.268598 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:07.269126 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.270315 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.271271 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:07.271650 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:07.271771 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:07.275526 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:07.275589 (+ 63us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:07.275590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:07.275702 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:07.278399 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:07.278802 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:07.278805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:07.278805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:07.278898 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:07.279104 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:07.279105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:07.279105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:07.279399 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:07.279550 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:07.280010 (+ 460us) server_negotiation.cc:299] Negotiation successful
0923 22:10:07.280156 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260923 22:10:07.290669 13922 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:07.291405 13922 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:07.297261 13922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Generated new TSK 0
I20260923 22:10:07.297432 13922 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:10:07.890918 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:07.989980 13953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:07.990155 13953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:07.990177 13953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:07.991264 13953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:07.991297 13953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:07.991319 13953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:07.992914 13953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:07.993115 13953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:07.993342 13953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:07.998534 13958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:07.998574 13961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:07.998545 13959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:07.999471 13953 server_base.cc:1034] running on GCE node
I20260923 22:10:07.999655 13953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:07.999938 13953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.001132 13953 hybrid_clock.cc:648] HybridClock initialized: now 1790201408001101 us; error 45 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.003981 13953 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:08.004396 13953 webserver.cc:460] Webserver started at http://127.1.161.129:33495/ using document root <none> and password file <none>
I20260923 22:10:08.004628 13953 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.004684 13953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:08.004815 13953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.021445 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "8af8dcc891d4485c8632775a9bf3e283"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "615f9ae045238420f2f8160faf614ac7"
server_key_iv: "19821a553ec504f68dda2cb33f9549da"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.021819 13953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "8af8dcc891d4485c8632775a9bf3e283"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "615f9ae045238420f2f8160faf614ac7"
server_key_iv: "19821a553ec504f68dda2cb33f9549da"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.023491 13953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:10:08.043308 13970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.043491 13953 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260923 22:10:08.043569 13953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "8af8dcc891d4485c8632775a9bf3e283"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "615f9ae045238420f2f8160faf614ac7"
server_key_iv: "19821a553ec504f68dda2cb33f9549da"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.043645 13953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:08.063908 13953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:08.065872 13953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:08.066000 13953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:08.066186 13953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:08.066509 13953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:08.066555 13953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.066591 13953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:08.066615 13953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.072371 13953 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:36791
I20260923 22:10:08.072854 13953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:08.075801 14083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:36791 every 8 connection(s)
I20260923 22:10:08.077360 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 13953
I20260923 22:10:08.077457 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:08.077649 1670 external_mini_cluster.cc:1428] Setting key 4b75b0ca6f09ae0ad8d23c25854b60ed
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:08.086352 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.076386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.129:42683)
0923 22:10:08.076627 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:08.076671 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:08.077310 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:08.077657 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:08.077661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:08.077879 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:08.078154 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.078160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.079145 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.079146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.080407 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.080410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.080977 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.080978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.081096 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.081565 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:08.081574 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:08.082987 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:08.084843 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.084849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.084851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.085100 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.085303 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.085304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.085304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.085422 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.085715 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:08.085722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:08.085894 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 22:10:08.085999 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 22:10:08.086134 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260923 22:10:08.086656 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.076426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:42683 (local address 127.1.161.190:41873)
0923 22:10:08.076559 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:08.076561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:08.077437 (+ 876us) server_negotiation.cc:407] Connection header received
0923 22:10:08.077467 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:08.077468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:08.077480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:08.077536 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:08.078262 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.079043 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.080503 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.080824 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.080896 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.083083 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:08.083084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:08.083084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:08.083091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:08.084758 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.085181 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.085182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.085182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.085239 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.085493 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.085494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.085494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.085613 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:08.085672 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:08.086481 (+ 809us) server_negotiation.cc:299] Negotiation successful
0923 22:10:08.086560 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260923 22:10:08.087124 14084 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:08.087227 14084 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.087373 14084 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:08.088060 13872 ts_manager.cc:194] Registered new tserver with Master: 8af8dcc891d4485c8632775a9bf3e283 (127.1.161.129:36791)
I20260923 22:10:08.088845 13872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:42683
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:10:08.115801 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 22:10:08.195986 14092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:08.196205 14092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:08.196275 14092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:08.197682 14092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:08.197739 14092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:08.197780 14092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:08.199620 14092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:08.199844 14092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:08.200090 14092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:08.204861 14100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.205099 14098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.205227 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:08.206245 14092 server_base.cc:1034] running on GCE node
I20260923 22:10:08.206416 14092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:08.206630 14092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.207794 14092 hybrid_clock.cc:648] HybridClock initialized: now 1790201408207771 us; error 48 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.210680 14092 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:08.211177 14092 webserver.cc:460] Webserver started at http://127.1.161.130:34209/ using document root <none> and password file <none>
I20260923 22:10:08.211362 14092 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.211392 14092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:08.211464 14092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.221045 14092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "b019791c2f2e41b28557d3f1dfdc91a4"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "8af8c8d35d69a19013ce6477ced467e7"
server_key_iv: "aea7469717a4ecb337addf09eaba48cb"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.221395 14092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "b019791c2f2e41b28557d3f1dfdc91a4"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "8af8c8d35d69a19013ce6477ced467e7"
server_key_iv: "aea7469717a4ecb337addf09eaba48cb"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.223665 14092 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260923 22:10:08.235291 14107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.235440 14092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260923 22:10:08.235513 14092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "b019791c2f2e41b28557d3f1dfdc91a4"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "8af8c8d35d69a19013ce6477ced467e7"
server_key_iv: "aea7469717a4ecb337addf09eaba48cb"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.235582 14092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:08.251492 14092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:08.253504 14092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:08.253616 14092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:08.253765 14092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:08.254065 14092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:08.254104 14092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.254129 14092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:08.254149 14092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.260533 14092 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:46711
I20260923 22:10:08.260908 14092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:08.261631 14221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:46711 every 8 connection(s)
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:08.269690 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.262159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:35713 (local address 127.1.161.190:41873)
0923 22:10:08.262201 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:08.262203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:08.262462 (+ 259us) server_negotiation.cc:407] Connection header received
0923 22:10:08.263013 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:08.263014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:08.263027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:08.263091 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:08.263726 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.264376 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.265200 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.265396 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.265472 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.267261 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:08.267262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:08.267263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:08.267270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:08.268452 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.268808 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.268809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.268810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.268862 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.269098 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.269100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.269100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.269198 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:08.269264 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:08.269527 (+ 263us) server_negotiation.cc:299] Negotiation successful
0923 22:10:08.269610 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:08.269833 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.262160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.130:35713)
0923 22:10:08.262320 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:08.262368 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:08.262904 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:08.263149 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:08.263152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:08.263405 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:08.263631 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.263636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.264460 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.264460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.265104 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.265106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.265492 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.265493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.265603 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.266067 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:08.266074 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:08.267195 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:08.268541 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.268546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.268548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.268729 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.268920 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.268921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.268922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.269026 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.269257 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:08.269262 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:08.269382 (+ 120us) client_negotiation.cc:769] Sending connection context
0923 22:10:08.269505 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 22:10:08.269620 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260923 22:10:08.270548 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14092
I20260923 22:10:08.270665 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:08.270857 1670 external_mini_cluster.cc:1428] Setting key a0d2e2f977438bba39e44e5de4fe4dcd
I20260923 22:10:08.271099 14222 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:08.271205 14222 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.271343 14222 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:08.271627 13872 ts_manager.cc:194] Registered new tserver with Master: b019791c2f2e41b28557d3f1dfdc91a4 (127.1.161.130:46711)
I20260923 22:10:08.272045 13872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:35713
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:10:08.294230 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:08.374858 14227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:08.375044 14227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:08.375095 14227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:08.376390 14227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:08.376441 14227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:08.376477 14227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:08.378641 14227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:08.378854 14227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:08.379068 14227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:08.384729 14232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.384858 14233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.385161 14235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:08.385161 14227 server_base.cc:1034] running on GCE node
I20260923 22:10:08.385413 14227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:08.385654 14227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.386801 14227 hybrid_clock.cc:648] HybridClock initialized: now 1790201408386792 us; error 25 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.389050 14227 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:08.389437 14227 webserver.cc:460] Webserver started at http://127.1.161.131:37131/ using document root <none> and password file <none>
I20260923 22:10:08.389638 14227 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.389698 14227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:08.389839 14227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.401741 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "6b72bff25e0b4f718984957c2b4f543d"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "71c6568afa8a5c6bc51a16f9613189b6"
server_key_iv: "d752539ff23ce055749be9bf1d416eee"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.402112 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "6b72bff25e0b4f718984957c2b4f543d"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "71c6568afa8a5c6bc51a16f9613189b6"
server_key_iv: "d752539ff23ce055749be9bf1d416eee"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.403659 14227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:10:08.415051 14242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.415206 14227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:08.415294 14227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "6b72bff25e0b4f718984957c2b4f543d"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "71c6568afa8a5c6bc51a16f9613189b6"
server_key_iv: "d752539ff23ce055749be9bf1d416eee"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.415364 14227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:08.429956 14227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:08.432077 14227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:08.432209 14227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:08.432418 14227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:08.432746 14227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:08.432785 14227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.432813 14227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:08.432833 14227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.446790 14227 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:37953
I20260923 22:10:08.446939 14355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:37953 every 8 connection(s)
I20260923 22:10:08.447149 14227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:08.448529 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14227
I20260923 22:10:08.448624 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:08.448801 1670 external_mini_cluster.cc:1428] Setting key 5bec7ca0d0a07641ef303cd34b1ba39c
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:08.456028 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.447867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:41893 (local address 127.1.161.190:41873)
0923 22:10:08.447906 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:08.447908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:08.448061 (+ 153us) server_negotiation.cc:407] Connection header received
0923 22:10:08.448644 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:08.448645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:08.448657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:08.448685 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:08.449397 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.450194 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.451567 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.451810 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.451849 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.453593 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:08.453594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:08.453594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:08.453600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:08.454775 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.455128 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.455129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.455129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.455202 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.455409 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.455411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.455411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.455518 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:08.455602 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:08.455871 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 22:10:08.455948 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:10:08.456161 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.447834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.131:41893)
0923 22:10:08.447953 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:08.447987 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:08.448537 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:08.448856 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:08.448860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:08.449079 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:08.449305 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.449310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.450301 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.450302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.451472 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.451486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.451922 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.451923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.452002 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.452451 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:08.452458 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:08.453512 (+ 1054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:08.454858 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.454863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.454865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.455050 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.455251 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.455252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.455252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.455348 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.455592 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:08.455597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:08.455723 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 22:10:08.455839 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:10:08.455961 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":63,"threads_started":1}
I20260923 22:10:08.456574 14356 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:08.456683 14356 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.456883 14356 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:08.457238 13872 ts_manager.cc:194] Registered new tserver with Master: 6b72bff25e0b4f718984957c2b4f543d (127.1.161.131:37953)
I20260923 22:10:08.457547 13872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:41893
I20260923 22:10:08.467756 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:08.468479 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 13953
I20260923 22:10:08.474294 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14092
I20260923 22:10:08.478240 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14227
I20260923 22:10:08.482365 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 13840
I20260923 22:10:08.486181 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41873
--webserver_interface=127.1.161.190
--webserver_port=38557
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41873
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:08.558161 14359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:08.558326 14359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:08.558344 14359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:08.559509 14359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:08.559545 14359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:08.559557 14359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:08.559569 14359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:08.559612 14359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:08.561094 14359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41873
--ranger_config_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=38557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:08.561299 14359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:08.561512 14359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:08.566075 14365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.566083 14367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.566090 14364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:08.566293 14359 server_base.cc:1034] running on GCE node
I20260923 22:10:08.566534 14359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:08.566802 14359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.567968 14359 hybrid_clock.cc:648] HybridClock initialized: now 1790201408567963 us; error 37 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.570230 14359 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:08.570653 14359 webserver.cc:460] Webserver started at http://127.1.161.190:38557/ using document root <none> and password file <none>
I20260923 22:10:08.570842 14359 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.570886 14359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.615728 14359 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:08.616458 14375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.616643 14359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:08.616740 14359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "46a5897d3c2147f09eab16e638d615c3"
format_stamp: "Formatted at 2026-09-23 22:10:07 on dist-test-slave-npvd"
server_key: "bd30cbb524971b9571646f548cbaf145"
server_key_iv: "22bc6c9e595d115f0a199a1a61f5e23f"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.617076 14359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:08.637036 14359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:08.638995 14359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:08.639132 14359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:08.643116 14359 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41873
I20260923 22:10:08.643209 14427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41873 every 8 connection(s)
I20260923 22:10:08.643553 14359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:08.644654 14428 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:08.645278 14428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Bootstrap starting.
I20260923 22:10:08.651355 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14359
I20260923 22:10:08.651700 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:36791
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:41873
--webserver_port=33495
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:08.653290 14428 log.cc:824] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:08.655148 14428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:08.655478 14428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Bootstrap complete.
I20260923 22:10:08.659394 14428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:08.659744 14428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a5897d3c2147f09eab16e638d615c3, State: Initialized, Role: FOLLOWER
I20260923 22:10:08.659950 14428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:08.660036 14428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:08.660091 14428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:08.660143 14428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:08.661621 14428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:08.661748 14428 leader_election.cc:302] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a5897d3c2147f09eab16e638d615c3; no voters:
I20260923 22:10:08.661906 14428 leader_election.cc:288] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:08.661993 14432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:08.662132 14432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [term 2 LEADER]: Becoming Leader. State: Replica: 46a5897d3c2147f09eab16e638d615c3, State: Running, Role: LEADER
I20260923 22:10:08.662155 14428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:08.662221 14432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } }
I20260923 22:10:08.662555 14433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46a5897d3c2147f09eab16e638d615c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } } }
I20260923 22:10:08.662628 14433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:08.662834 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a5897d3c2147f09eab16e638d615c3. Latest consensus state: current_term: 2 leader_uuid: "46a5897d3c2147f09eab16e638d615c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a5897d3c2147f09eab16e638d615c3" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41873 } } }
I20260923 22:10:08.663005 14440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:08.663056 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:08.663543 14440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:08.663664 14440 catalog_manager.cc:1259] Loaded cluster ID: 17f94dfa4a33470c8949039269982e6b
I20260923 22:10:08.663707 14440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:08.663990 14428 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 22:10:08.664722 14440 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:08.664911 14440 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Loaded TSK: 0
I20260923 22:10:08.665112 14440 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:08.762260 14430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:08.762511 14430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:08.762558 14430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:08.764019 14430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:08.764075 14430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:08.764110 14430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:08.765967 14430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:36791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=33495
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:08.766170 14430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:08.766393 14430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:08.772482 14465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.772992 14466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.773185 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:08.773371 14430 server_base.cc:1034] running on GCE node
I20260923 22:10:08.773564 14430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:08.773797 14430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.774973 14430 hybrid_clock.cc:648] HybridClock initialized: now 1790201408774946 us; error 37 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.778108 14430 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:08.778575 14430 webserver.cc:460] Webserver started at http://127.1.161.129:33495/ using document root <none> and password file <none>
I20260923 22:10:08.778802 14430 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.778868 14430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.802557 14430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:08.803489 14476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.803686 14430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:08.803752 14430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "8af8dcc891d4485c8632775a9bf3e283"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "615f9ae045238420f2f8160faf614ac7"
server_key_iv: "19821a553ec504f68dda2cb33f9549da"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.804134 14430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:08.822167 14430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:08.839833 14430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:08.840037 14430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:08.840301 14430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:08.840678 14430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:08.840764 14430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.840852 14430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:08.840919 14430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.847694 14430 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:36791
I20260923 22:10:08.848208 14430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:08.849570 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14430
I20260923 22:10:08.849885 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:46711
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:41873
--webserver_port=34209
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:08.853282 14589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:36791 every 8 connection(s)
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:08.866261 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.853821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.129:60341)
0923 22:10:08.854012 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:08.854052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:08.855663 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:08.856391 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:08.856395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:08.856611 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:08.856853 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.856859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.858352 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.858353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.859546 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:08.859549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.860463 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.860464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:08.860589 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.861187 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:08.861205 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:08.862709 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:08.864765 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.864771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.864773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.864957 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.865216 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:08.865217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:08.865217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:08.865351 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:08.865754 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:08.865760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:08.865884 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 22:10:08.865981 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:10:08.866101 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260923 22:10:08.866700 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:08.854163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:60341 (local address 127.1.161.190:41873)
0923 22:10:08.854498 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:08.854507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:08.854542 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:10:08.855929 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:08.855936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:08.855989 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:08.856110 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:08.856973 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.858231 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.859637 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:08.860252 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:08.860355 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:08.862823 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:08.862827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:08.862828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:08.862838 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:08.864686 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.865045 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.865047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.865048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.865137 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:08.865448 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:08.865449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:08.865449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:08.865682 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:08.865770 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:08.866433 (+ 663us) server_negotiation.cc:299] Negotiation successful
0923 22:10:08.866562 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":85,"threads_started":1}
I20260923 22:10:08.867321 14590 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:08.867408 14590 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.867532 14590 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:08.868373 14590 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:41873 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:08.868436 14590 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.868492 14590 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:08.868793 14590 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:08.868861 14590 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:08.869124 14590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:08.961655 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:08.961881 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:08.961910 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:08.963416 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:08.963464 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:08.963496 14592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:08.965401 14592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:46711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=34209
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:08.965652 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:08.965894 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:08.970731 14599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.970745 14600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:08.970995 14602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:08.971534 14592 server_base.cc:1034] running on GCE node
I20260923 22:10:08.971726 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:08.971992 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:08.979295 14592 hybrid_clock.cc:648] HybridClock initialized: now 1790201408979269 us; error 37 us; skew 500 ppm
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:08.981745 14592 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:08.982151 14592 webserver.cc:460] Webserver started at http://127.1.161.130:34209/ using document root <none> and password file <none>
I20260923 22:10:08.982357 14592 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:08.982401 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:08 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:08.996857 14592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:08.997740 14610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:08.997937 14592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:08.998087 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "b019791c2f2e41b28557d3f1dfdc91a4"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "8af8c8d35d69a19013ce6477ced467e7"
server_key_iv: "aea7469717a4ecb337addf09eaba48cb"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:08.998594 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:09.011914 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:09.014153 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:09.014289 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:09.014508 14592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:09.014816 14592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:09.014863 14592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.014891 14592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:09.014906 14592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.021886 14592 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:46711
I20260923 22:10:09.021968 14723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:46711 every 8 connection(s)
I20260923 22:10:09.022295 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201408, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:09.029004 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14592
I20260923 22:10:09.029292 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.161.148:39587/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:37953
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:41873
--webserver_port=37131
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:09.032140 14726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.022943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.130:56589)
0923 22:10:09.023282 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.023322 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.023748 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.023971 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.023974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.024125 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.024292 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.024296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.024979 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.024979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.025664 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.025666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.026020 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.026021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.026120 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.026534 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.026541 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.027929 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.030570 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.030577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.030579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.030857 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.031107 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.031108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.031109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.031248 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.031556 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.031561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.031694 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.031778 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.031906 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":271,"threads_started":1}
I20260923 22:10:09.032574 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.022955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:56589 (local address 127.1.161.190:41873)
0923 22:10:09.023013 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.023015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.023392 (+ 377us) server_negotiation.cc:407] Connection header received
0923 22:10:09.023850 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.023852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.023866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.023911 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.024384 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.024897 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.025743 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.025933 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.025999 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.028027 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.028028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.028028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.028034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.030470 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.030947 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.030949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.030949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.031023 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.031334 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.031335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.031335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.031463 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.031520 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.032404 (+ 884us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.032483 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:10:09.032881 14724 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:09.032984 14724 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.033169 14724 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:09.033797 14724 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:41873 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:09.033864 14724 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.033921 14724 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:09.034188 14724 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.034250 14724 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:09.034487 14724 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:09.139287 14727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:09.139539 14727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:09.139573 14727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:09.141194 14727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:09.141244 14727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:09.141278 14727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:09.143083 14727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.161.148:39587/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:37953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=37131
--tserver_master_addrs=127.1.161.190:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:09.143297 14727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:09.143584 14727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:09.149191 14733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.149220 14736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:09.149335 14734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:09.149654 14727 server_base.cc:1034] running on GCE node
I20260923 22:10:09.149850 14727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:09.150077 14727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:09.151221 14727 hybrid_clock.cc:648] HybridClock initialized: now 1790201409151202 us; error 33 us; skew 500 ppm
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:09.153482 14727 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:09.153909 14727 webserver.cc:460] Webserver started at http://127.1.161.131:37131/ using document root <none> and password file <none>
I20260923 22:10:09.154114 14727 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:09.154181 14727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:09.172226 14727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:09.173136 14744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.173317 14727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:09.173394 14727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "6b72bff25e0b4f718984957c2b4f543d"
format_stamp: "Formatted at 2026-09-23 22:10:08 on dist-test-slave-npvd"
server_key: "71c6568afa8a5c6bc51a16f9613189b6"
server_key_iv: "d752539ff23ce055749be9bf1d416eee"
server_key_version: "kuduclusterkey@0"
I20260923 22:10:09.173748 14727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:09.195868 14727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:09.197654 14727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:09.197793 14727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:09.198057 14727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:09.198523 14727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:09.198577 14727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.198613 14727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:09.198637 14727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:09.205148 14727 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:37953
I20260923 22:10:09.205502 14727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:09.206051 14857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:37953 every 8 connection(s)
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:09.211695 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14727
I20260923 22:10:09.215108 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.206269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:47685 (local address 127.1.161.190:41873)
0923 22:10:09.206309 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.206311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.206524 (+ 213us) server_negotiation.cc:407] Connection header received
0923 22:10:09.206968 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.206969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.206981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.207036 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.207665 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.208167 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.209373 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.209641 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.209714 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.212411 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.212412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.212413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.212420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.213795 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.214227 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.214228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.214229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.214285 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.214514 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.214515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.214515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.214638 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.214709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.214951 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.215023 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:10:09.215327 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.206285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:41873 (local address 127.1.161.131:47685)
0923 22:10:09.206420 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:09.206449 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:09.206876 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:09.207093 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:09.207096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:09.207320 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:09.207559 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.207565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.208243 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.208244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.209288 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:09.209291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.209749 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.209750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:09.209904 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.210433 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:09.210442 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:09.212288 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:09.213870 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.213877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.213886 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.214150 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.214362 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:09.214363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:09.214363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:09.214451 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:09.214685 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:09.214690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:09.214833 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 22:10:09.214932 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:10:09.215061 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260923 22:10:09.215658 14858 heartbeater.cc:347] Connected to a master server at 127.1.161.190:41873
I20260923 22:10:09.215739 14858 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.215875 14858 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:09.216430 14858 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:41873 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 22:10:09.216498 14858 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.216567 14858 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:09.216881 14858 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.216950 14858 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
W20260923 22:10:09.217471 14858 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:09.219125 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:09.212085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.1.161.190:41873)
0923 22:10:09.212242 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:09.212244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:09.212431 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:10:09.212454 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:09.212456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:09.212467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:09.212526 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:09.213739 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.214253 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.215406 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:09.215656 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:09.215707 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:09.216935 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:09.216936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:09.216936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:09.216942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:09.218229 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.218565 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.218566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.218566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.218606 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:09.218775 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:09.218776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:09.218776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:09.218844 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:09.218932 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:09.219026 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:10:09.219073 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":53,"threads_started":1}
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:09.869311 14590 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:09.869460 14590 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:09.869792 14392 ts_manager.cc:194] Registered new tserver with Master: 8af8dcc891d4485c8632775a9bf3e283 (127.1.161.129:36791)
I20260923 22:10:09.870812 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:60341
Sep 23 22:10:09 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:10.034641 14724 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:10.034770 14724 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:10.035019 14392 ts_manager.cc:194] Registered new tserver with Master: b019791c2f2e41b28557d3f1dfdc91a4 (127.1.161.130:46711)
I20260923 22:10:10.035418 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:56589
Sep 23 22:10:10 dist-test-slave-npvd krb5kdc[13396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for HTTP/127.1.161.148@KRBTEST.COM
I20260923 22:10:10.217620 14858 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:10.217746 14858 heartbeater.cc:511] Master 127.1.161.190:41873 requested a full tablet report, sending...
I20260923 22:10:10.218071 14392 ts_manager.cc:194] Registered new tserver with Master: 6b72bff25e0b4f718984957c2b4f543d (127.1.161.131:37953)
I20260923 22:10:10.218523 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:47685
I20260923 22:10:10.223956 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:10.241510 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.226655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.1.161.190:41873)
0923 22:10:10.226751 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.226753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.226843 (+ 90us) server_negotiation.cc:407] Connection header received
0923 22:10:10.226889 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.226889 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.226905 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.226966 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:10.227657 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.228312 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.233239 (+ 4927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.233551 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.233590 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.238606 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:10.238607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:10.238608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:10.238615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:10.240227 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.240645 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.240646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.240647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.240710 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:10.240938 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:10.240939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:10.240939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:10.241070 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:10.241171 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.241348 (+ 177us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.241426 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
I20260923 22:10:10.243551 14392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:10.245637 14392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:10.275655 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:37953 (local address 127.0.0.1:38972)
0923 22:10:10.263964 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.263999 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.264272 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.271555 (+ 7283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.271558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.271560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:10.271640 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.271642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.273041 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.273042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.274208 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.274246 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.275389 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.275390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.275468 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.275480 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.275536 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.275578 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":5,"thread_start_us":60,"threads_started":1}
I20260923 22:10:10.276211 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.1.161.131:37953)
0923 22:10:10.271324 (+ 7728us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.271329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.271339 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:10.271371 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.271373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.271388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.271462 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:10.271715 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.272943 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.274322 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.275250 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.275310 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.275360 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.276104 (+ 744us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.276141 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7695,"thread_start_us":56,"threads_started":1}
I20260923 22:10:10.276736 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:36791 (local address 127.0.0.1:52718)
0923 22:10:10.264453 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.264459 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.264483 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.271562 (+ 7079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.271563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.271566 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:10.271631 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.271633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.272752 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.272752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.274248 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.274251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.276537 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.276538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.276589 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.276593 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.276643 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.276673 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":400,"thread_start_us":43,"threads_started":1}
I20260923 22:10:10.276984 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.1.161.129:36791)
0923 22:10:10.271323 (+ 7368us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.271328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.271339 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:10.271373 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.271375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.271390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.271470 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:10.271706 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.272663 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.275738 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.276397 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.276461 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.276507 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.276673 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.276713 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7332,"thread_start_us":59,"threads_started":1}
I20260923 22:10:10.276966 14780 tablet_service.cc:1467] Processing CreateTablet for tablet 99ed590ef1c1438185a335d032af907e (DEFAULT_TABLE table=test-table [id=94c2b7d80ed242b1b10b749334fb8a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:10.277298 14780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ed590ef1c1438185a335d032af907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:10.277680 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 99ed590ef1c1438185a335d032af907e (DEFAULT_TABLE table=test-table [id=94c2b7d80ed242b1b10b749334fb8a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:10.277982 14524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ed590ef1c1438185a335d032af907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:10.278846 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:46711 (local address 127.0.0.1:49606)
0923 22:10:10.264365 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.264372 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.264399 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.273435 (+ 9036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.273436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.273438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:10.273474 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.273475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.276843 (+ 3368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.276844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.277774 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.277776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.278641 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.278642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.278695 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.278698 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.278757 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.278789 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":55,"threads_started":1}
I20260923 22:10:10.279157 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.263772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.1.161.130:46711)
0923 22:10:10.273203 (+ 9431us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.273207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.273217 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:10:10.273247 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.273249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.273262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.273351 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:10.274347 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.275659 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.277860 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.278551 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.278975 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.279035 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.279058 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.279094 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9397,"thread_start_us":46,"threads_started":1}
I20260923 22:10:10.279675 14658 tablet_service.cc:1467] Processing CreateTablet for tablet 99ed590ef1c1438185a335d032af907e (DEFAULT_TABLE table=test-table [id=94c2b7d80ed242b1b10b749334fb8a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:10.279942 14658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ed590ef1c1438185a335d032af907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:10.281905 14895 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Bootstrap starting.
I20260923 22:10:10.282459 14895 tablet_bootstrap.cc:654] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.282758 14895 log.cc:824] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.283478 14895 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: No bootstrap required, opened a new log
I20260923 22:10:10.283536 14895 ts_tablet_manager.cc:1397] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:10.286057 14895 raft_consensus.cc:348] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.286185 14895 raft_consensus.cc:374] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.286211 14895 raft_consensus.cc:729] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af8dcc891d4485c8632775a9bf3e283, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.286665 14895 consensus_queue.cc:260] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.287024 14895 ts_tablet_manager.cc:1428] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:10:10.287168 14590 heartbeater.cc:503] Master 127.1.161.190:41873 was elected leader, sending a full tablet report...
I20260923 22:10:10.287354 14896 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: Bootstrap starting.
I20260923 22:10:10.287976 14896 tablet_bootstrap.cc:654] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.295346 14894 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Bootstrap starting.
I20260923 22:10:10.295486 14896 log.cc:824] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.296049 14894 tablet_bootstrap.cc:654] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:10.296267 14896 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: No bootstrap required, opened a new log
I20260923 22:10:10.296360 14894 log.cc:824] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:10.296396 14896 ts_tablet_manager.cc:1397] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260923 22:10:10.297118 14894 tablet_bootstrap.cc:492] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: No bootstrap required, opened a new log
I20260923 22:10:10.297178 14894 ts_tablet_manager.cc:1397] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:10.298887 14896 raft_consensus.cc:348] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.299029 14896 raft_consensus.cc:374] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.299063 14896 raft_consensus.cc:729] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b019791c2f2e41b28557d3f1dfdc91a4, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.299217 14896 consensus_queue.cc:260] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.299242 14894 raft_consensus.cc:348] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.299374 14894 raft_consensus.cc:374] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:10.299402 14894 raft_consensus.cc:729] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b72bff25e0b4f718984957c2b4f543d, State: Initialized, Role: FOLLOWER
I20260923 22:10:10.299458 14896 ts_tablet_manager.cc:1428] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:10:10.299494 14894 consensus_queue.cc:260] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.299557 14724 heartbeater.cc:503] Master 127.1.161.190:41873 was elected leader, sending a full tablet report...
I20260923 22:10:10.299710 14894 ts_tablet_manager.cc:1428] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:10:10.299989 14858 heartbeater.cc:503] Master 127.1.161.190:41873 was elected leader, sending a full tablet report...
W20260923 22:10:10.355378 14591 tablet.cc:2369] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:10.447548 14902 raft_consensus.cc:482] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:10.447691 14902 raft_consensus.cc:504] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.448032 14902 leader_election.cc:288] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8af8dcc891d4485c8632775a9bf3e283 (127.1.161.129:36791), b019791c2f2e41b28557d3f1dfdc91a4 (127.1.161.130:46711)
I20260923 22:10:10.451059 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.448276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:39141 (local address 127.1.161.129:36791)
0923 22:10:10.448331 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.448333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.448533 (+ 200us) server_negotiation.cc:407] Connection header received
0923 22:10:10.448634 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.448635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.448644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.448671 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:10.448849 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.449318 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.450327 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.450690 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.450775 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.450794 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.450952 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.450991 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:10:10.451074 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.448195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:36791 (local address 127.1.161.131:39141)
0923 22:10:10.448466 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.448476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.448559 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.448733 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.448733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.448735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:10.448784 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.448786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.449399 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.449400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.450219 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.450222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.450789 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.450790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.450857 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.450861 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.450942 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.450981 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":76,"threads_started":1}
I20260923 22:10:10.451422 14544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ed590ef1c1438185a335d032af907e" candidate_uuid: "6b72bff25e0b4f718984957c2b4f543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8dcc891d4485c8632775a9bf3e283" is_pre_election: true
I20260923 22:10:10.451575 14544 raft_consensus.cc:2454] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b72bff25e0b4f718984957c2b4f543d in term 0.
I20260923 22:10:10.451794 14747 leader_election.cc:302] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b72bff25e0b4f718984957c2b4f543d, 8af8dcc891d4485c8632775a9bf3e283; no voters:
I20260923 22:10:10.452045 14902 raft_consensus.cc:2781] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:10.452128 14902 raft_consensus.cc:482] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:10.452159 14902 raft_consensus.cc:3037] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.452648 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.448368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:54349 (local address 127.1.161.130:46711)
0923 22:10:10.448424 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.448426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.448438 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:10.448540 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.448541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.448553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.448606 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:10.449249 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.449961 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.451721 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.452278 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.452330 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.452338 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.452543 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.452584 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:10:10.452836 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.448192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:46711 (local address 127.1.161.131:54349)
0923 22:10:10.448381 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.448391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:10.448464 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:10.448625 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:10.448626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:10.448628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:10.449145 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.449147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.450065 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.450066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.451252 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:10.451254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.452395 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.452396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:10.452463 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.452467 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:10.452725 (+ 258us) client_negotiation.cc:240] Negotiation successful
0923 22:10:10.452765 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260923 22:10:10.452858 14902 raft_consensus.cc:504] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.452973 14902 leader_election.cc:288] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [CANDIDATE]: Term 1 election: Requested vote from peers 8af8dcc891d4485c8632775a9bf3e283 (127.1.161.129:36791), b019791c2f2e41b28557d3f1dfdc91a4 (127.1.161.130:46711)
I20260923 22:10:10.453161 14544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ed590ef1c1438185a335d032af907e" candidate_uuid: "6b72bff25e0b4f718984957c2b4f543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af8dcc891d4485c8632775a9bf3e283"
I20260923 22:10:10.453238 14544 raft_consensus.cc:3037] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.453224 14678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ed590ef1c1438185a335d032af907e" candidate_uuid: "6b72bff25e0b4f718984957c2b4f543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" is_pre_election: true
I20260923 22:10:10.453372 14678 raft_consensus.cc:2454] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b72bff25e0b4f718984957c2b4f543d in term 0.
I20260923 22:10:10.453486 14677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ed590ef1c1438185a335d032af907e" candidate_uuid: "6b72bff25e0b4f718984957c2b4f543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b019791c2f2e41b28557d3f1dfdc91a4"
I20260923 22:10:10.453541 14677 raft_consensus.cc:3037] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:10.454032 14544 raft_consensus.cc:2454] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b72bff25e0b4f718984957c2b4f543d in term 1.
I20260923 22:10:10.454181 14747 leader_election.cc:302] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b72bff25e0b4f718984957c2b4f543d, 8af8dcc891d4485c8632775a9bf3e283; no voters:
I20260923 22:10:10.454257 14902 raft_consensus.cc:2781] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:10.454277 14677 raft_consensus.cc:2454] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b72bff25e0b4f718984957c2b4f543d in term 1.
I20260923 22:10:10.454373 14902 raft_consensus.cc:686] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 1 LEADER]: Becoming Leader. State: Replica: 6b72bff25e0b4f718984957c2b4f543d, State: Running, Role: LEADER
I20260923 22:10:10.454460 14902 consensus_queue.cc:237] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } }
I20260923 22:10:10.455250 14391 catalog_manager.cc:5495] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b72bff25e0b4f718984957c2b4f543d (127.1.161.131). New cstate: current_term: 1 leader_uuid: "6b72bff25e0b4f718984957c2b4f543d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b72bff25e0b4f718984957c2b4f543d" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 } health_report { overall_health: UNKNOWN } } }
W20260923 22:10:10.457326 14859 tablet.cc:2369] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:10.501135 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.498679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.1.161.131:37953)
0923 22:10:10.498716 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:10.498717 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:10.498725 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:10.498794 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:10.498794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:10.498808 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:10.498861 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:10.498989 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.499441 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.500443 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:10.500619 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:10.500669 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:10.500839 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:10.500930 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:10.500983 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:10.501042 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:10:10.501057 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:10:10.502810 14677 raft_consensus.cc:1261] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 1 FOLLOWER]: Refusing update from remote peer 6b72bff25e0b4f718984957c2b4f543d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:10.502861 14544 raft_consensus.cc:1261] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 1 FOLLOWER]: Refusing update from remote peer 6b72bff25e0b4f718984957c2b4f543d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:10.503154 14902 consensus_queue.cc:1035] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af8dcc891d4485c8632775a9bf3e283" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:10.503293 14912 consensus_queue.cc:1035] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b019791c2f2e41b28557d3f1dfdc91a4" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 46711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:10.505431 14915 mvcc.cc:205] Tried to move back new op lower bound from 7332664977416859648 to 7332664977221885952. Current Snapshot: MvccSnapshot[applied={T|T < 7332664977416859648}]
I20260923 22:10:10.505640 14916 mvcc.cc:205] Tried to move back new op lower bound from 7332664977416859648 to 7332664977221885952. Current Snapshot: MvccSnapshot[applied={T|T < 7332664977416859648}]
I20260923 22:10:10.507225 14917 mvcc.cc:205] Tried to move back new op lower bound from 7332664977416859648 to 7332664977221885952. Current Snapshot: MvccSnapshot[applied={T|T < 7332664977416859648 or (T in {7332664977416859648})}]
I20260923 22:10:10.507889 14391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:10.508001 14391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:10.508751 14391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46a5897d3c2147f09eab16e638d615c3: Sending DeleteTablet for 3 replicas of tablet 99ed590ef1c1438185a335d032af907e
I20260923 22:10:10.509116 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ed590ef1c1438185a335d032af907e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:10 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:52718
I20260923 22:10:10.509392 14924 tablet_replica.cc:330] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: stopping tablet replica
I20260923 22:10:10.509399 14658 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ed590ef1c1438185a335d032af907e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:10 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:49606
I20260923 22:10:10.509563 14924 raft_consensus.cc:2229] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:10.509694 14924 raft_consensus.cc:2258] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:10.510121 14780 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ed590ef1c1438185a335d032af907e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:10 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38972
I20260923 22:10:10.510260 14924 ts_tablet_manager.cc:1905] T 99ed590ef1c1438185a335d032af907e P 8af8dcc891d4485c8632775a9bf3e283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:10.510643 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14430
I20260923 22:10:10.517819 14926 tablet_replica.cc:330] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: stopping tablet replica
I20260923 22:10:10.517969 14926 raft_consensus.cc:2229] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 1 LEADER]: Raft consensus shutting down.
W20260923 22:10:10.517946 14378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:36791 (error 108)
I20260923 22:10:10.518117 14926 raft_consensus.cc:2258] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:10.518492 14925 tablet_replica.cc:330] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4: stopping tablet replica
I20260923 22:10:10.518591 14926 ts_tablet_manager.cc:1905] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:10.518641 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.518506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:36791 (local address 127.0.0.1:52720)
0923 22:10:10.518553 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.518592 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:36791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:10.518748 14378 catalog_manager.cc:4584] TS 8af8dcc891d4485c8632775a9bf3e283 (127.1.161.129:36791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99ed590ef1c1438185a335d032af907e: Network error: Client connection negotiation failed: client connection to 127.1.161.129:36791: connect: Connection refused (error 111)
I20260923 22:10:10.518749 14925 raft_consensus.cc:2229] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:10.518936 14925 raft_consensus.cc:2258] T 99ed590ef1c1438185a335d032af907e P b019791c2f2e41b28557d3f1dfdc91a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:10.518973 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14592
I20260923 22:10:10.521054 14926 ts_tablet_manager.cc:1918] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:10.521126 14926 log.cc:1196] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/99ed590ef1c1438185a335d032af907e
I20260923 22:10:10.521397 14926 ts_tablet_manager.cc:1939] T 99ed590ef1c1438185a335d032af907e P 6b72bff25e0b4f718984957c2b4f543d: Deleting consensus metadata
I20260923 22:10:10.521713 14376 catalog_manager.cc:4841] TS 6b72bff25e0b4f718984957c2b4f543d (127.1.161.131:37953): tablet 99ed590ef1c1438185a335d032af907e (table test-table [id=94c2b7d80ed242b1b10b749334fb8a4c]) successfully deleted
I20260923 22:10:10.525576 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:10.525462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:46711 (local address 127.0.0.1:49620)
0923 22:10:10.525505 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:10.525540 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.130:46711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 22:10:10.525792 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14727
I20260923 22:10:10.529875 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14359
I20260923 22:10:10.535161 1670 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 22:10:10.869004 1670 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 22:10:10.869082 1670 mini_ranger.cc:66] Stopping Ranger...
I20260923 22:10:11.213568 1670 mini_ranger.cc:68] Stopped Ranger
2026-09-23 22:10:11.213 UTC [13426] LOG: received smart shutdown request
2026-09-23 22:10:11.217 UTC [13426] LOG: background worker "logical replication launcher" (PID 13435) exited with exit code 1
2026-09-23 22:10:11.217 UTC [13430] LOG: shutting down
2026-09-23 22:10:11.367 UTC [13426] LOG: database system is shut down
2026-09-23T22:10:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (42457 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:11 dist-test-slave-npvd krb5kdc[14936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:11 dist-test-slave-npvd krb5kdc[14936](info): set up 2 sockets
Sep 23 22:10:11 dist-test-slave-npvd krb5kdc[14936](info): commencing operation
krb5kdc: starting...
W20260923 22:10:12.402630 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:10:12.502908 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44313
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:12.573572 14949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:12.573729 14949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:12.573750 14949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:12.574721 14949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:12.574757 14949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:12.574770 14949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:12.574816 14949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:12.574838 14949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:12.576289 14949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:12.576536 14949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:12.576725 14949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:12.581089 14957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.581141 14949 server_base.cc:1034] running on GCE node
W20260923 22:10:12.581074 14955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:12.581074 14954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.581619 14949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:12.581887 14949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:12.583042 14949 hybrid_clock.cc:648] HybridClock initialized: now 1790201412583030 us; error 42 us; skew 500 ppm
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:12.585449 14949 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:12.585878 14949 webserver.cc:460] Webserver started at http://127.1.161.190:40645/ using document root <none> and password file <none>
I20260923 22:10:12.586083 14949 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:12.586123 14949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:12.586200 14949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:12.587069 14949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "a04886203b9f4e3e838af8d0ec9703f6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "4288029c7e12a9de90f460f1800ded12"
server_key_iv: "146286b7f4624ef8ec42cdcaec15dd87"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.587422 14949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "a04886203b9f4e3e838af8d0ec9703f6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "4288029c7e12a9de90f460f1800ded12"
server_key_iv: "146286b7f4624ef8ec42cdcaec15dd87"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.588554 14949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:12.589195 14964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:12.589394 14949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:12.589458 14949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "a04886203b9f4e3e838af8d0ec9703f6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "4288029c7e12a9de90f460f1800ded12"
server_key_iv: "146286b7f4624ef8ec42cdcaec15dd87"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.589543 14949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:12.602802 14949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:12.604473 14949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:12.604624 14949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:12.608655 14949 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:44313
I20260923 22:10:12.608682 15016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:44313 every 8 connection(s)
I20260923 22:10:12.608979 14949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:12.609679 15017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:12.611013 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:12.616549 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 14949
I20260923 22:10:12.616649 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:10:12.616834 1670 external_mini_cluster.cc:1428] Setting key 68a228b6543883f4bade4adbaa27c738
I20260923 22:10:12.619555 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.611718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44313 (local address 127.0.0.1:52426)
0923 22:10:12.611873 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.611901 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.612273 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.612565 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.612567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.612797 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.612933 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.612937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.613824 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.613825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.614438 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.614439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.614829 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.614830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.614878 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.615293 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.615300 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.616449 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.618076 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.618081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.618083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.618299 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.618546 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.618547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.618547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.618632 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.618975 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.618980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.619037 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.619141 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.619386 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":82,"thread_start_us":85,"threads_started":1}
I20260923 22:10:12.619563 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.611735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.1.161.190:44313)
0923 22:10:12.612398 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.612401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.612410 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:12.612459 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.612460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.612472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.612528 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.613015 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.613754 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.614502 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.614762 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.614830 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.616527 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.616535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.616535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.616542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.617953 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.618379 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.618382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.618382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.618476 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.618698 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.618699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.618699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.618892 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.618982 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.619190 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.619330 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":73,"threads_started":1}
I20260923 22:10:12.620148 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.620482 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.620369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37108)
0923 22:10:12.620415 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.620454 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 22:10:12.620635 14967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
I20260923 22:10:12.621061 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.620975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37120)
0923 22:10:12.621023 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.621046 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:10:12.621178 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:39707: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.161.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
I20260923 22:10:12.639847 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:39707
--webserver_interface=127.1.161.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:12.670387 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } attempt: 1
I20260923 22:10:12.670784 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.670660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37134)
0923 22:10:12.670722 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.670758 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:10:12.671108 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.671014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37136)
0923 22:10:12.671059 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.671085 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:10:12.671217 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:39707: Network error: Client connection negotiation failed: client connection to 127.1.161.189:39707: connect: Connection refused (error 111)
W20260923 22:10:12.710686 15024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:12.710866 15024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:12.710888 15024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:12.711994 15024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:12.712030 15024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:12.712044 15024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:12.712054 15024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:12.712100 15024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:12.713346 15024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.161.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:12.713562 15024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:12.713749 15024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:12.717697 15032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:12.717721 15030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.717813 15024 server_base.cc:1034] running on GCE node
W20260923 22:10:12.717689 15029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.718250 15024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:12.718511 15024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:12.719645 15024 hybrid_clock.cc:648] HybridClock initialized: now 1790201412719636 us; error 26 us; skew 500 ppm
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:12.721873 15024 init.cc:377] Logged in from keytab as kudu/127.1.161.189@KRBTEST.COM (short username kudu)
I20260923 22:10:12.722275 15024 webserver.cc:460] Webserver started at http://127.1.161.189:34061/ using document root <none> and password file <none>
I20260923 22:10:12.722470 15024 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:12.722529 15024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:12.722661 15024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:12.723703 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data/instance:
uuid: "4435b32fbc654805b11d4c5c21e98964"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "736b6f8727bafc9309508c733b89ed83"
server_key_iv: "96a4768ae00d40ffbdb36772dcbf1c8a"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.724027 15024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal/instance:
uuid: "4435b32fbc654805b11d4c5c21e98964"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "736b6f8727bafc9309508c733b89ed83"
server_key_iv: "96a4768ae00d40ffbdb36772dcbf1c8a"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.725225 15024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:10:12.725845 15039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:12.725996 15024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:12.726084 15024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "4435b32fbc654805b11d4c5c21e98964"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "736b6f8727bafc9309508c733b89ed83"
server_key_iv: "96a4768ae00d40ffbdb36772dcbf1c8a"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.726183 15024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:12.732304 15024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:12.733750 15024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:12.733881 15024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:12.737423 15024 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.189:39707
I20260923 22:10:12.737449 15091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.189:39707 every 8 connection(s)
I20260923 22:10:12.737722 15024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
I20260923 22:10:12.738440 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:12.739358 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } attempt: 2
I20260923 22:10:12.739888 15092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.743439 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15024
I20260923 22:10:12.743541 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-1/wal/instance
I20260923 22:10:12.743726 1670 external_mini_cluster.cc:1428] Setting key 594145ad0d90d6b9237aa65911a3c7a9
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:12.748427 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.739602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37148)
0923 22:10:12.739665 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.739675 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.739723 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.740565 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.740566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.740739 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.740794 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.740796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.741765 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.741765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.742408 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.742409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.743055 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.743055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.743126 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.743655 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.743661 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.745114 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.747146 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.747147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.747147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.747352 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.747627 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.747628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.747629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.747695 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.748108 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.748109 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.748155 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.748230 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.748316 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 22:10:12.748646 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.739739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.1.161.189:39707)
0923 22:10:12.739891 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.739898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.739917 (+ 19us) server_negotiation.cc:407] Connection header received
0923 22:10:12.740353 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.740358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.740401 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.740568 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.740881 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.741705 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.742467 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.742980 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.743109 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.745213 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.745221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.745222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.745229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.747072 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.747438 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.747441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.747442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.747536 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.747766 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.747767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.747767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.748033 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.748112 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.748304 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.748425 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260923 22:10:12.749019 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.740787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.1.161.190:44313)
0923 22:10:12.740856 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.740858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.741021 (+ 163us) server_negotiation.cc:407] Connection header received
0923 22:10:12.741080 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.741080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.741089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.741143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.741623 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.742265 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.743223 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.743523 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.743588 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.746333 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.746334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.746334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.746341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.747875 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.748265 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.748266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.748267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.748323 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.748511 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.748511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.748512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.748639 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.748673 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.748879 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.748945 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:10:12.749037 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.740723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44313 (local address 127.0.0.1:52436)
0923 22:10:12.740921 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.740959 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.741028 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.741162 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.741165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.741422 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.741520 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.741523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.742336 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.742337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.743121 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.743123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.743598 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.743599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.744146 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.744604 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.744610 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.746028 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.747949 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.747955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.747958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.748189 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.748390 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.748391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.748391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.748444 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.748713 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.748719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.748761 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.748851 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.748945 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":100,"threads_started":1}
I20260923 22:10:12.749341 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.749622 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.749529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58622)
0923 22:10:12.749575 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.749602 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:10:12.749714 15092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.750725 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.750626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58628)
0923 22:10:12.750677 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.750701 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 22:10:12.750850 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:41319: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.161.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.188@KRBTEST.COM" created.
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:10:12.757169 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.749929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37154)
0923 22:10:12.749977 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.749985 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.750018 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.750136 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.750136 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.750296 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.750327 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.750328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.751259 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.751260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.751991 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.751994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.752291 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.752292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.752328 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.752610 (+ 282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.752613 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.754847 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.756318 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.756319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.756319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.756469 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.756654 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.756654 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.756655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.756697 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.756938 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.756938 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.756966 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.757009 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.757109 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 22:10:12.757301 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.749953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.1.161.189:39707)
0923 22:10:12.749982 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.749984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.750009 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:10:12.750058 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.750059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.750067 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.750130 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.750397 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.751147 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.752058 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.752232 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.752283 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.754945 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.754946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.754946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.754951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.756245 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.756536 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.756537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.756538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.756592 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.756751 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.756752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.756752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.756881 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.756943 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.757034 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.757171 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":13}
I20260923 22:10:12.757682 15092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.757948 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.757857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58636)
0923 22:10:12.757896 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.757930 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:10:12.758095 15042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
I20260923 22:10:12.758438 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.758359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58640)
0923 22:10:12.758397 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.758419 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:10:12.758534 15092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:41319: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
I20260923 22:10:12.767580 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:41319
--webserver_interface=127.1.161.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:12.793965 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } attempt: 1
I20260923 22:10:12.794286 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.794191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58642)
0923 22:10:12.794238 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.794265 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:10:12.794574 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.794507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58650)
0923 22:10:12.794541 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.794560 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:10:12.794656 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:41319: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
I20260923 22:10:12.807754 15092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } attempt: 1
I20260923 22:10:12.808142 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.808049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58658)
0923 22:10:12.808092 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.808120 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 22:10:12.808454 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.808377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58670)
0923 22:10:12.808416 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.808438 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:10:12.808557 15092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:41319: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
W20260923 22:10:12.837718 15099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:12.837867 15099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:12.837885 15099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:12.839288 15099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:12.839342 15099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:12.839365 15099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:12.839388 15099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:12.839425 15099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:12.840857 15099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44313,127.1.161.189:39707,127.1.161.188:41319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:41319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.161.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:12.841095 15099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:12.841288 15099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:10:12.841773 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } attempt: 2
I20260923 22:10:12.842140 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.842018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58674)
0923 22:10:12.842078 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.842119 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 22:10:12.842417 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.842350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58680)
0923 22:10:12.842380 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.842399 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 22:10:12.842495 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:41319: Network error: Client connection negotiation failed: client connection to 127.1.161.188:41319: connect: Connection refused (error 111)
W20260923 22:10:12.845567 15104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:12.845561 15105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.845593 15099 server_base.cc:1034] running on GCE node
W20260923 22:10:12.845559 15107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:12.845907 15099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:12.846103 15099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:12.847265 15099 hybrid_clock.cc:648] HybridClock initialized: now 1790201412847247 us; error 33 us; skew 500 ppm
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:12.849392 15099 init.cc:377] Logged in from keytab as kudu/127.1.161.188@KRBTEST.COM (short username kudu)
I20260923 22:10:12.849748 15099 webserver.cc:460] Webserver started at http://127.1.161.188:33545/ using document root <none> and password file <none>
I20260923 22:10:12.849954 15099 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:12.849992 15099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:12.850118 15099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:12.851068 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data/instance:
uuid: "da26de90331f45f7b0376af380d9eff6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "7d41ef103b97b7f07d2e2f1487590e07"
server_key_iv: "979f4273ed5b293f7e579dfb7e404850"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.851486 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal/instance:
uuid: "da26de90331f45f7b0376af380d9eff6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "7d41ef103b97b7f07d2e2f1487590e07"
server_key_iv: "979f4273ed5b293f7e579dfb7e404850"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.852578 15099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:12.853166 15114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:12.853327 15099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:12.853415 15099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "da26de90331f45f7b0376af380d9eff6"
format_stamp: "Formatted at 2026-09-23 22:10:12 on dist-test-slave-npvd"
server_key: "7d41ef103b97b7f07d2e2f1487590e07"
server_key_iv: "979f4273ed5b293f7e579dfb7e404850"
server_key_version: "encryptionkey@0"
I20260923 22:10:12.853477 15099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:12.868197 15099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:12.869728 15099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:12.869867 15099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:12.873730 15099 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.188:41319
I20260923 22:10:12.873777 15166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.188:41319 every 8 connection(s)
I20260923 22:10:12.874055 15099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
I20260923 22:10:12.874727 15167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:12.875852 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.876704 15092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } attempt: 2
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:10:12.881013 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15099
I20260923 22:10:12.881129 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790201274647240-1670-0/minicluster-data/master-2/wal/instance
I20260923 22:10:12.881309 1670 external_mini_cluster.cc:1428] Setting key 576bc53a11bd9dda5704053ead73242d
I20260923 22:10:12.884124 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.876337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.1.161.190:44313)
0923 22:10:12.876412 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.876414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.876577 (+ 163us) server_negotiation.cc:407] Connection header received
0923 22:10:12.877080 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.877081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.877091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.877138 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.877582 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.877991 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.878757 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.878947 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.879014 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.881002 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.881003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.881004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.881011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.882668 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883103 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.883105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.883105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.883199 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883449 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.883449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.883450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.883581 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.883648 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.883934 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.884008 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:10:12.884207 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.876320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44313 (local address 127.0.0.1:52448)
0923 22:10:12.876469 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.876495 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.876964 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.877165 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.877169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.877380 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.877499 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.877502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.878063 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.878064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.878674 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.878676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.879029 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.879029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.879151 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.879600 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.879607 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.880915 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.882747 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.882753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.882756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.883026 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.883266 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.883267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.883382 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.883650 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.883655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.883780 (+ 125us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.883877 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.884001 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260923 22:10:12.884737 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.876904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58696)
0923 22:10:12.876950 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.876958 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.876998 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.877248 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.877249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.877399 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.877434 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.877436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.878247 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.878248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.878697 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.878699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.879130 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.879131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.879317 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.879743 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.879748 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.881307 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.883520 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.883521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.883708 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.883977 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.883978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.884030 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.884474 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.884475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.884515 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.884580 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.884658 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:10:12.884842 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:10:12.885066 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.876946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.1.161.188:41319)
0923 22:10:12.877084 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.877087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.877110 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:10:12.877157 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.877158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.877171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.877251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.877509 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.878167 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.878770 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.879050 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.879138 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.881423 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.881431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.881431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.881438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.883453 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.883791 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.883794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.883794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.883845 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.884118 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.884118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.884119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.884409 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.884467 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.884897 (+ 430us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.884964 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260923 22:10:12.886631 15092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: Bootstrap starting.
I20260923 22:10:12.887519 15092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: Neither blocks nor log segments found. Creating new log.
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:10:12.887769 15092 log.cc:824] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: Log is configured to *not* fsync() on all Append() calls
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:10:12.888424 15092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: No bootstrap required, opened a new log
I20260923 22:10:12.890110 15092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.890232 15092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:12.890304 15092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4435b32fbc654805b11d4c5c21e98964, State: Initialized, Role: FOLLOWER
I20260923 22:10:12.890457 15092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.890771 15092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:12.890738 15174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:12.890947 15174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:12.891297 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.885032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:39707 (local address 127.0.0.1:37164)
0923 22:10:12.885122 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.885131 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.885171 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.885264 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.885264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.885390 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.885431 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.885432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.886006 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.886006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.886471 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.886473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.886795 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.886796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.886855 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.887203 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.887207 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.888345 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.889974 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.889974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.889975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.890179 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.890486 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.890486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.890487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.890547 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.890872 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.890872 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.890913 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.890984 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.891070 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260923 22:10:12.891942 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.885089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.1.161.189:39707)
0923 22:10:12.885129 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.885131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.885165 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:10:12.885210 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.885211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.885219 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.885260 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.885501 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.885907 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.886561 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.886730 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.886795 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.888429 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.888430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.888430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.888436 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.889898 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.890341 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.890342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.890343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.890400 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.890622 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.890624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.890624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.890763 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.890842 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.891754 (+ 912us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.891833 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:12.892150 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.881702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.1.161.188:41319)
0923 22:10:12.881829 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.881831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.882255 (+ 424us) server_negotiation.cc:407] Connection header received
0923 22:10:12.882325 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.882325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.882334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.882413 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.883476 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.884109 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.885097 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.885638 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.885685 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.888538 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.888539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.888540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.888545 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.890745 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.891140 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.891141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.891141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.891228 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.891437 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.891438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.891438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.891562 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.891966 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.892003 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.892066 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260923 22:10:12.892184 15167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
W20260923 22:10:12.893517 15185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:12.893604 15185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:12.893730 15024 master_runner.cc:186] Error getting master registration for 127.1.161.190:44313: OK, instance_id { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" instance_seqno: 1790201412607492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:10:12.896534 15024 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:10:12.901060 15169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.892730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58716)
0923 22:10:12.892793 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.892802 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.892834 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.893357 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.893357 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.893473 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.893497 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.893498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.894215 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.894215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.894822 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.894824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.895271 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.895271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.895331 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.895646 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.895650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.897519 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.899993 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.899994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.899994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.900171 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.900423 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.900424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.900424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.900475 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.900738 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.900739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.900774 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.900864 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.900954 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 22:10:12.901331 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.893159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.1.161.188:41319)
0923 22:10:12.893245 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.893247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.893253 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:10:12.893270 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.893271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.893280 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.893329 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.893561 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.894139 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.894903 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.895157 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.895229 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.897602 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.897602 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.897603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.897609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.899922 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.900235 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.900236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.900237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.900309 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.900536 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.900537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.900537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.900659 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.900733 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.900886 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.901242 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":284,"server-negotiator.queue_time_us":70}
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:12.902477 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: Bootstrap starting.
I20260923 22:10:12.903014 15167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:12.903308 15167 log.cc:824] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:12.903937 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: No bootstrap required, opened a new log
I20260923 22:10:12.904352 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.895273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.1.161.190:44313)
0923 22:10:12.896720 (+ 1447us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.896721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.896728 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:12.896744 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.896745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.896751 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.896787 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.897292 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.897883 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.898824 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.899112 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.899576 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.901948 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.901949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.901949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.901955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.903405 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.903700 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.903701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.903701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.903749 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.903927 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.903928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.903929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.904033 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.904162 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.904192 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.904249 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412}
I20260923 22:10:12.905669 15167 raft_consensus.cc:348] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.905807 15167 raft_consensus.cc:374] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:12.905841 15167 raft_consensus.cc:729] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da26de90331f45f7b0376af380d9eff6, State: Initialized, Role: FOLLOWER
I20260923 22:10:12.906002 15167 consensus_queue.cc:260] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.906257 15188 sys_catalog.cc:455] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:12.906299 15167 sys_catalog.cc:564] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:12.906441 15188 sys_catalog.cc:458] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: This master's current role is: FOLLOWER
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
W20260923 22:10:12.908349 15199 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:12.908974 15199 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:12.909188 15099 master_runner.cc:186] Error getting master registration for 127.1.161.190:44313: OK, instance_id { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" instance_seqno: 1790201412607492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:10:12.911651 15099 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:10:12.911947 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.904760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.1.161.189:39707)
0923 22:10:12.904793 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.904794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.904832 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:10:12.904891 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.904892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.904899 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.904939 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.905388 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.905798 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.906644 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.906835 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.906909 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.909125 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.909126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.909126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.909132 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.910608 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.910981 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.910982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.910982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.911035 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.911253 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.911254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.911254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.911377 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.911516 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.911830 (+ 314us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.911891 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 22:10:12.913316 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:10:12.921612 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.914786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.1.161.190:44313)
0923 22:10:12.914887 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.914888 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.914895 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:12.914912 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.914912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.914919 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.914961 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.915576 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.916181 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.917063 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.917318 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.917353 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.918746 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.918747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.918747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.918753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.920314 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.920699 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.920700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.920700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.920751 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.921009 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.921010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.921011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.921133 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.921201 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.921416 (+ 215us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.921531 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 22:10:12.922144 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.915004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.1.161.189:39707)
0923 22:10:12.915070 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.915071 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.915077 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:10:12.915093 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.915093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.915100 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.915145 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.915711 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.916260 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.917063 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.917328 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.917358 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.918843 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.918844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.918844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.918849 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.920744 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.921128 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.921130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.921130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.921178 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.921394 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.921395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.921395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.921548 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.921602 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.921980 (+ 378us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.922041 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 22:10:12.927830 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.915041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.1.161.188:41319)
0923 22:10:12.915203 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.915205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.918186 (+ 2981us) server_negotiation.cc:407] Connection header received
0923 22:10:12.922599 (+ 4413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.922600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.922609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.922633 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.923320 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.923707 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.924253 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.924427 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.924491 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.925536 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.925537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.925538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.925543 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.926763 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.927081 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.927082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.927082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.927128 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.927371 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.927371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.927371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.927466 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.927571 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.927661 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.927724 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260923 22:10:12.949625 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } attempt: 3
Sep 23 22:10:12 dist-test-slave-npvd krb5kdc[14936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201412, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:10:12.955240 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.949921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.1.161.188:41319)
0923 22:10:12.949959 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:12.949960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:12.949968 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:12.949989 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:12.949990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:12.949997 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:12.950058 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:12.950352 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.950773 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.951508 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.951669 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.951732 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.953132 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:12.953134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:12.953134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:12.953140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:12.954278 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.954575 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.954576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.954577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.954626 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:12.954807 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:12.954807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:12.954808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:12.954894 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:12.954959 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:12.955101 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:10:12.955154 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:10:12.955267 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:12.949871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:41319 (local address 127.0.0.1:58732)
0923 22:10:12.949911 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:12.949919 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:12.949951 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:12.950062 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:12.950062 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:12.950205 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:12.950235 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.950237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.950852 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.950854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.951379 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:12.951381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:12.951730 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:12.951731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:12.951786 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:12.952119 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:12.952123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:12.953058 (+ 935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:12.954368 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.954370 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.954370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.954513 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.954689 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:12.954689 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:12.954690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:12.954737 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:12.954968 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:12.954969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:12.955018 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 22:10:12.955080 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:10:12.955154 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:10:12.956655 15017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: Bootstrap starting.
I20260923 22:10:12.957360 15017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:12.957616 15017 log.cc:824] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:12.958534 15017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: No bootstrap required, opened a new log
I20260923 22:10:12.959892 15017 raft_consensus.cc:348] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.960004 15017 raft_consensus.cc:374] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:12.960071 15017 raft_consensus.cc:729] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04886203b9f4e3e838af8d0ec9703f6, State: Initialized, Role: FOLLOWER
I20260923 22:10:12.960202 15017 consensus_queue.cc:260] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:12.960489 15209 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:12.960544 15017 sys_catalog.cc:564] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:12.960613 15209 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:10:12.962249 15220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:10:12.962332 15220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:10:12.964063 14949 master_runner.cc:263] No leader master found from master a04886203b9f4e3e838af8d0ec9703f6
I20260923 22:10:12.964102 14949 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 22:10:12.984397 15045 tablet.cc:2369] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:13.081512 15188 raft_consensus.cc:482] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:13.081621 15188 raft_consensus.cc:504] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:13.081837 15188 leader_election.cc:288] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a04886203b9f4e3e838af8d0ec9703f6 (127.1.161.190:44313), 4435b32fbc654805b11d4c5c21e98964 (127.1.161.189:39707)
I20260923 22:10:13.082056 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da26de90331f45f7b0376af380d9eff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4435b32fbc654805b11d4c5c21e98964" is_pre_election: true
I20260923 22:10:13.082080 14991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da26de90331f45f7b0376af380d9eff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a04886203b9f4e3e838af8d0ec9703f6" is_pre_election: true
I20260923 22:10:13.082162 15066 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da26de90331f45f7b0376af380d9eff6 in term 0.
I20260923 22:10:13.082199 14991 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da26de90331f45f7b0376af380d9eff6 in term 0.
I20260923 22:10:13.082322 15117 leader_election.cc:302] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4435b32fbc654805b11d4c5c21e98964, da26de90331f45f7b0376af380d9eff6; no voters:
I20260923 22:10:13.082440 15188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:13.082497 15188 raft_consensus.cc:482] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:13.082511 15188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:13.083122 15188 raft_consensus.cc:504] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:13.083264 15188 leader_election.cc:288] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [CANDIDATE]: Term 1 election: Requested vote from peers a04886203b9f4e3e838af8d0ec9703f6 (127.1.161.190:44313), 4435b32fbc654805b11d4c5c21e98964 (127.1.161.189:39707)
I20260923 22:10:13.083369 14991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da26de90331f45f7b0376af380d9eff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a04886203b9f4e3e838af8d0ec9703f6"
I20260923 22:10:13.083400 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da26de90331f45f7b0376af380d9eff6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4435b32fbc654805b11d4c5c21e98964"
I20260923 22:10:13.083452 14991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:13.083460 15066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:13.084021 14991 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da26de90331f45f7b0376af380d9eff6 in term 1.
I20260923 22:10:13.084043 15066 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da26de90331f45f7b0376af380d9eff6 in term 1.
I20260923 22:10:13.084204 15115 leader_election.cc:302] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a04886203b9f4e3e838af8d0ec9703f6, da26de90331f45f7b0376af380d9eff6; no voters:
I20260923 22:10:13.084285 15188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:13.084456 15188 raft_consensus.cc:686] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [term 1 LEADER]: Becoming Leader. State: Replica: da26de90331f45f7b0376af380d9eff6, State: Running, Role: LEADER
I20260923 22:10:13.084554 15188 consensus_queue.cc:237] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } }
I20260923 22:10:13.084884 15222 sys_catalog.cc:455] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da26de90331f45f7b0376af380d9eff6. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.084964 15222 sys_catalog.cc:458] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:13.085134 15224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:13.085307 15224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:13.086385 14991 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [term 1 FOLLOWER]: Refusing update from remote peer da26de90331f45f7b0376af380d9eff6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:13.086445 15066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [term 1 FOLLOWER]: Refusing update from remote peer da26de90331f45f7b0376af380d9eff6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:13.086717 15188 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:13.086799 15222 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:13.087852 15209 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da26de90331f45f7b0376af380d9eff6. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.087926 15209 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:13.087882 15174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da26de90331f45f7b0376af380d9eff6. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.087947 15174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:13.088591 15188 sys_catalog.cc:455] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.088660 15188 sys_catalog.cc:458] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:13.088698 15188 sys_catalog.cc:455] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.088744 15188 sys_catalog.cc:458] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:13.088804 15225 mvcc.cc:205] Tried to move back new op lower bound from 7332664987999911936 to 7332664987994783744. Current Snapshot: MvccSnapshot[applied={T|T < 7332664987999911936}]
I20260923 22:10:13.088786 15174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.088820 15226 mvcc.cc:205] Tried to move back new op lower bound from 7332664987999911936 to 7332664987994783744. Current Snapshot: MvccSnapshot[applied={T|T < 7332664987999911936}]
I20260923 22:10:13.088845 15174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:13.088923 15209 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da26de90331f45f7b0376af380d9eff6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a04886203b9f4e3e838af8d0ec9703f6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44313 } } peers { permanent_uuid: "4435b32fbc654805b11d4c5c21e98964" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 39707 } } peers { permanent_uuid: "da26de90331f45f7b0376af380d9eff6" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 41319 } } }
I20260923 22:10:13.088984 15209 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:10:13.089121 15224 catalog_manager.cc:1347] Generated new cluster ID: 2569f5da1673459eaa8dc508ae1056db
I20260923 22:10:13.089164 15224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:13.097460 15224 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:13.098049 15224 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:13.109931 15224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da26de90331f45f7b0376af380d9eff6: Generated new TSK 0
I20260923 22:10:13.110105 15224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:10:13.894104 15185 catalog_manager.cc:1259] Loaded cluster ID: 2569f5da1673459eaa8dc508ae1056db
I20260923 22:10:13.894181 15185 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: loading cluster ID for follower catalog manager: success
I20260923 22:10:13.895203 15185 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: acquiring CA information for follower catalog manager: success
I20260923 22:10:13.895543 15185 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4435b32fbc654805b11d4c5c21e98964: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:10:13.962721 15220 catalog_manager.cc:1259] Loaded cluster ID: 2569f5da1673459eaa8dc508ae1056db
I20260923 22:10:13.962786 15220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: loading cluster ID for follower catalog manager: success
I20260923 22:10:13.963454 15220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: acquiring CA information for follower catalog manager: success
I20260923 22:10:13.963670 15220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a04886203b9f4e3e838af8d0ec9703f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:10:14.160339 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 14949
I20260923 22:10:14.165187 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15024
I20260923 22:10:14.169611 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15099
2026-09-23T22:10:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2703 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:14 dist-test-slave-npvd krb5kdc[15240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:14 dist-test-slave-npvd krb5kdc[15240](info): set up 2 sockets
Sep 23 22:10:14 dist-test-slave-npvd krb5kdc[15240](info): commencing operation
krb5kdc: starting...
W20260923 22:10:15.122766 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:10:15.222744 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38965
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:15.291749 15253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:15.291901 15253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:15.291919 15253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:15.292867 15253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:15.292896 15253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:15.292908 15253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:15.292919 15253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:15.292930 15253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:15.294128 15253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:15.294273 15253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:15.294405 15253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:15.298403 15258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.298411 15261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:15.298447 15259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:15.298725 15253 server_base.cc:1034] running on GCE node
I20260923 22:10:15.298965 15253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:15.299269 15253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:15.300427 15253 hybrid_clock.cc:648] HybridClock initialized: now 1790201415300425 us; error 38 us; skew 500 ppm
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:15.307359 15253 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:15.307729 15253 webserver.cc:460] Webserver started at http://127.1.161.190:43869/ using document root <none> and password file <none>
I20260923 22:10:15.307934 15253 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:15.307994 15253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:15.308130 15253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:15.309015 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "e776a4afed044b73bd6de60ff4d9ace7"
format_stamp: "Formatted at 2026-09-23 22:10:15 on dist-test-slave-npvd"
server_key: "ef67041cdaba1c2b947286e17c93f924"
server_key_iv: "63dccfa9c757f6c84cad9cce0052a26d"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.309319 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "e776a4afed044b73bd6de60ff4d9ace7"
format_stamp: "Formatted at 2026-09-23 22:10:15 on dist-test-slave-npvd"
server_key: "ef67041cdaba1c2b947286e17c93f924"
server_key_iv: "63dccfa9c757f6c84cad9cce0052a26d"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.310415 15253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:10:15.311071 15268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:15.311281 15253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:15.311355 15253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "e776a4afed044b73bd6de60ff4d9ace7"
format_stamp: "Formatted at 2026-09-23 22:10:15 on dist-test-slave-npvd"
server_key: "ef67041cdaba1c2b947286e17c93f924"
server_key_iv: "63dccfa9c757f6c84cad9cce0052a26d"
server_key_version: "encryptionkey@0"
I20260923 22:10:15.311439 15253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:15.322633 15253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:15.324126 15253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:15.324262 15253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:15.327853 15253 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38965
I20260923 22:10:15.327881 15320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38965 every 8 connection(s)
I20260923 22:10:15.328174 15253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:15.328913 15321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:15.330888 15321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7: Bootstrap starting.
I20260923 22:10:15.331453 15321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:15.331761 15321 log.cc:824] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:15.332446 15321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7: No bootstrap required, opened a new log
I20260923 22:10:15.333876 15321 raft_consensus.cc:348] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } }
I20260923 22:10:15.333995 15321 raft_consensus.cc:374] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:15.334014 15321 raft_consensus.cc:729] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e776a4afed044b73bd6de60ff4d9ace7, State: Initialized, Role: FOLLOWER
I20260923 22:10:15.334165 15321 consensus_queue.cc:260] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } }
I20260923 22:10:15.334221 15321 raft_consensus.cc:388] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:15.334240 15321 raft_consensus.cc:482] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:15.334264 15321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:15.334725 15321 raft_consensus.cc:504] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } }
I20260923 22:10:15.334821 15321 leader_election.cc:302] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e776a4afed044b73bd6de60ff4d9ace7; no voters:
I20260923 22:10:15.334995 15321 leader_election.cc:288] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:15.335181 15324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:15.335403 15321 sys_catalog.cc:564] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:15.335387 15324 raft_consensus.cc:686] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [term 1 LEADER]: Becoming Leader. State: Replica: e776a4afed044b73bd6de60ff4d9ace7, State: Running, Role: LEADER
I20260923 22:10:15.335556 15324 consensus_queue.cc:237] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } }
I20260923 22:10:15.336220 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15253
I20260923 22:10:15.336220 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e776a4afed044b73bd6de60ff4d9ace7. Latest consensus state: current_term: 1 leader_uuid: "e776a4afed044b73bd6de60ff4d9ace7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } } }
I20260923 22:10:15.336293 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:15.336337 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:10:15.336342 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e776a4afed044b73bd6de60ff4d9ace7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e776a4afed044b73bd6de60ff4d9ace7" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38965 } } }
I20260923 22:10:15.336395 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:15.336510 1670 external_mini_cluster.cc:1428] Setting key c54d2e36f0903601be58accb56b9d30e
I20260923 22:10:15.336900 15333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:15.337440 15333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:15.338920 15333 catalog_manager.cc:1347] Generated new cluster ID: c3ea127a80fa4318985ca57f2f877fa3
I20260923 22:10:15.338972 15333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:15.346407 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:15.337103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.1.161.190:38965)
0923 22:10:15.337342 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:15.337352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:15.337386 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:10:15.337502 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:15.337509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:15.337562 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:15.337705 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:15.338423 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.339589 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.340288 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:15.340715 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:15.340847 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:15.342970 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:15.342973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:15.342974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:15.342984 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:15.344952 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.345320 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.345322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.345323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.345422 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:15.345616 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:15.345617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:15.345617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:15.345815 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:15.345972 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:15.346042 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:10:15.346175 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":64,"threads_started":1}
I20260923 22:10:15.347218 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:10:15.353044 15333 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:15.353472 15333 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:15.358304 15333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e776a4afed044b73bd6de60ff4d9ace7: Generated new TSK 0
I20260923 22:10:15.358446 15333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:10:15.400599 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15253
2026-09-23T22:10:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1229 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15348](info): set up 2 sockets
Sep 23 22:10:15 dist-test-slave-npvd krb5kdc[15348](info): commencing operation
krb5kdc: starting...
W20260923 22:10:16.326855 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.909s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:10:16.434230 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:16.506134 15361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:16.506300 15361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:16.506319 15361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:16.507378 15361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:16.507412 15361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:16.507424 15361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:16.507436 15361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:16.507474 15361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:16.508723 15361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:16.508879 15361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:16.509079 15361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:16.513398 15369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.513365 15367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.513383 15366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.513651 15361 server_base.cc:1034] running on GCE node
I20260923 22:10:16.513844 15361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:16.514036 15361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:16.515156 15361 hybrid_clock.cc:648] HybridClock initialized: now 1790201416515133 us; error 41 us; skew 500 ppm
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:16.517355 15361 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:16.517730 15361 webserver.cc:460] Webserver started at http://127.1.161.190:39241/ using document root <none> and password file <none>
I20260923 22:10:16.517935 15361 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:16.517975 15361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:16.518100 15361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:16.519028 15361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "65759b0ed96e49328c4d4b690a74829d"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "ed32da83ea4c18e8b8c059c9066a4829"
server_key_iv: "0c657f6c3fea18e7e5bc943ceb2a7289"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.519384 15361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "65759b0ed96e49328c4d4b690a74829d"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "ed32da83ea4c18e8b8c059c9066a4829"
server_key_iv: "0c657f6c3fea18e7e5bc943ceb2a7289"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.520537 15361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:16.521225 15376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.521608 15361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:16.521677 15361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "65759b0ed96e49328c4d4b690a74829d"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "ed32da83ea4c18e8b8c059c9066a4829"
server_key_iv: "0c657f6c3fea18e7e5bc943ceb2a7289"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.521761 15361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.529567 15361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.531315 15361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.531459 15361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.534884 15361 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:44727
I20260923 22:10:16.534948 15428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:44727 every 8 connection(s)
I20260923 22:10:16.535279 15361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:16.536087 15429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:16.537851 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15361
I20260923 22:10:16.538005 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:10:16.538187 15429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap starting.
I20260923 22:10:16.538210 1670 external_mini_cluster.cc:1428] Setting key c718f0a9c06632c292ea73e32c406203
I20260923 22:10:16.538882 15429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:16.539268 15429 log.cc:824] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:16.539965 15429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: No bootstrap required, opened a new log
I20260923 22:10:16.541728 15429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:16.541877 15429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:16.541898 15429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Initialized, Role: FOLLOWER
I20260923 22:10:16.542039 15429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:16.542148 15429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:16.542205 15429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:16.542254 15429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:16.542774 15429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:16.542881 15429 leader_election.cc:302] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65759b0ed96e49328c4d4b690a74829d; no voters:
I20260923 22:10:16.543049 15429 leader_election.cc:288] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:16.543071 15434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:16.543208 15434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 LEADER]: Becoming Leader. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Running, Role: LEADER
I20260923 22:10:16.543335 15429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:16.543344 15434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:16.543777 15436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65759b0ed96e49328c4d4b690a74829d. Latest consensus state: current_term: 1 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:16.543864 15436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:16.544234 15435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:16.544371 15435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:16.544567 15442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:16.545022 15442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:16.546525 15442 catalog_manager.cc:1347] Generated new cluster ID: e0f4b566caeb4d2f8beca683e906aacb
I20260923 22:10:16.546581 15442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:16.548130 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.538667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.1.161.190:44727)
0923 22:10:16.538837 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:16.538845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:16.538877 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:10:16.539453 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:16.539459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:16.539499 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:16.539642 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:16.540366 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.541210 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.542008 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.542463 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.542565 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.545061 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:16.545065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:16.545065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:16.545076 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:16.546835 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.547138 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.547141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.547141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.547246 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.547427 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.547428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.547428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.547622 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:16.547735 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:16.547847 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:10:16.547959 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260923 22:10:16.568607 15442 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:16.569244 15442 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:16.577327 15442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Generated new TSK 0
I20260923 22:10:16.577466 15442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:10:16.605352 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:16.680019 15455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:16.680226 15455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:16.680270 15455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:16.681403 15455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:16.681468 15455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:16.681514 15455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:16.682834 15455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:16.683060 15455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:16.683305 15455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:16.687551 15460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.687564 15463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.687593 15461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.687798 15455 server_base.cc:1034] running on GCE node
I20260923 22:10:16.688017 15455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:16.688251 15455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:16.689407 15455 hybrid_clock.cc:648] HybridClock initialized: now 1790201416689397 us; error 43 us; skew 500 ppm
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:16.691694 15455 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:16.692107 15455 webserver.cc:460] Webserver started at http://127.1.161.129:45625/ using document root <none> and password file <none>
I20260923 22:10:16.692319 15455 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:16.692360 15455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:16.692489 15455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:16.693369 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "3b31d999354c9e785f6a2b9884dc1325"
server_key_iv: "4f019ff5626a1016eec4c260d8c4110a"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.693676 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "3b31d999354c9e785f6a2b9884dc1325"
server_key_iv: "4f019ff5626a1016eec4c260d8c4110a"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.694849 15455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:16.695591 15470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.695779 15455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.695860 15455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "3b31d999354c9e785f6a2b9884dc1325"
server_key_iv: "4f019ff5626a1016eec4c260d8c4110a"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.695936 15455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.708190 15455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.709695 15455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.709828 15455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.710027 15455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:16.710304 15455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:16.710333 15455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.710350 15455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:16.710361 15455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.715734 15455 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37407
I20260923 22:10:16.715777 15583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37407 every 8 connection(s)
I20260923 22:10:16.716008 15455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:16.719357 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15455
I20260923 22:10:16.719442 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:16.719633 1670 external_mini_cluster.cc:1428] Setting key 111bf3b31f66b452754001b2aef6390f
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:16.724908 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.716536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:55465 (local address 127.1.161.190:44727)
0923 22:10:16.716571 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:16.716573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:16.716747 (+ 174us) server_negotiation.cc:407] Connection header received
0923 22:10:16.717132 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:16.717133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:16.717141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:16.717188 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:16.717945 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.718476 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.719564 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.719843 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.719921 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.722044 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:16.722045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:16.722045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:16.722051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:16.723520 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.723958 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.723959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.723960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.724018 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.724280 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.724281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.724281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.724392 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:16.724468 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:16.724738 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 22:10:16.724821 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:10:16.725095 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.716510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:55465)
0923 22:10:16.716645 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.716679 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:16.717058 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:16.717256 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:16.717259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:16.717727 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:16.717876 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.717879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.718571 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.718572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.719493 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.719496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.719928 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.719929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.720043 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.720469 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:16.720477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:16.721954 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:16.723634 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.723640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.723642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:16.723856 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:16.724092 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.724093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.724093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:16.724199 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:16.724474 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:16.724480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:16.724604 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 22:10:16.724704 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:10:16.724858 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260923 22:10:16.725706 15584 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:16.725803 15584 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:16.725947 15584 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:16.726504 15393 ts_manager.cc:194] Registered new tserver with Master: a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407)
I20260923 22:10:16.727397 15393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:55465
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:10:16.743345 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:16.814368 15589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:16.814522 15589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:16.814541 15589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:16.815521 15589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:16.815555 15589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:16.815577 15589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:16.816953 15589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:16.817173 15589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:16.817394 15589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:16.821825 15595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.821817 15597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.821859 15594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.822031 15589 server_base.cc:1034] running on GCE node
I20260923 22:10:16.822259 15589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:16.822438 15589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:16.823601 15589 hybrid_clock.cc:648] HybridClock initialized: now 1790201416823588 us; error 22 us; skew 500 ppm
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:16.825933 15589 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:16.826428 15589 webserver.cc:460] Webserver started at http://127.1.161.130:39799/ using document root <none> and password file <none>
I20260923 22:10:16.826700 15589 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:16.826776 15589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:16.826901 15589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:16.827888 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "60b29f5163f41e4a95f15e4b21e44f10"
server_key_iv: "c4d8dae3bc782f8d0de1c8cfb27541b6"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.828231 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "60b29f5163f41e4a95f15e4b21e44f10"
server_key_iv: "c4d8dae3bc782f8d0de1c8cfb27541b6"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.829439 15589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:16.830050 15604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.830207 15589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:16.830286 15589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "60b29f5163f41e4a95f15e4b21e44f10"
server_key_iv: "c4d8dae3bc782f8d0de1c8cfb27541b6"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.830355 15589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.853816 15589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.855993 15589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.856143 15589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.856344 15589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:16.856607 15589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:16.856665 15589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.856704 15589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:16.856732 15589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.862968 15589 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:35431
I20260923 22:10:16.863018 15717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:35431 every 8 connection(s)
I20260923 22:10:16.863346 15589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:16.867312 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15589
I20260923 22:10:16.867394 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:16.867542 1670 external_mini_cluster.cc:1428] Setting key 4a98b57b49de3460bfdb74610bce653a
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:16.871882 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.863919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:45735 (local address 127.1.161.190:44727)
0923 22:10:16.863981 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:16.863984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:16.864159 (+ 175us) server_negotiation.cc:407] Connection header received
0923 22:10:16.864597 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:16.864598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:16.864607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:16.864651 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:16.865378 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.865787 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.866636 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.866812 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.866861 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.868890 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:16.868891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:16.868891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:16.868899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:16.870477 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.870880 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.870882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.870882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.870941 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.871218 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.871218 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.871219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.871331 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:16.871400 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:16.871705 (+ 305us) server_negotiation.cc:299] Negotiation successful
0923 22:10:16.871807 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:10:16.872025 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.863894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:45735)
0923 22:10:16.864040 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.864068 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:16.864516 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:16.864705 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:16.864708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:16.865157 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:16.865309 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.865312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.865856 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.865857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.866560 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.866561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.866883 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.866884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.866973 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.867500 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:16.867513 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:16.868804 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:16.870564 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.870570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.870572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:16.870800 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:16.871014 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.871015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.871015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:16.871115 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:16.871404 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:16.871410 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:16.871557 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 22:10:16.871683 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 22:10:16.871840 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260923 22:10:16.872372 15718 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:16.872447 15718 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:16.872593 15718 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:16.872876 15393 ts_manager.cc:194] Registered new tserver with Master: c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
I20260923 22:10:16.873248 15393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:45735
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:10:16.889645 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:16.960412 15723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:16.960604 15723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:16.960656 15723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:16.961732 15723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:16.961807 15723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:16.961858 15723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:16.963086 15723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:16.963332 15723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:16.963518 15723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:16.967801 15729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:16.967810 15728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.967908 15723 server_base.cc:1034] running on GCE node
W20260923 22:10:16.967816 15731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:16.968201 15723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:16.968377 15723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:16.969553 15723 hybrid_clock.cc:648] HybridClock initialized: now 1790201416969543 us; error 23 us; skew 500 ppm
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:16.972034 15723 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:16.972402 15723 webserver.cc:460] Webserver started at http://127.1.161.131:39411/ using document root <none> and password file <none>
I20260923 22:10:16.972564 15723 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:16.972599 15723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:16.972672 15723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:16.973492 15723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "91f236bd7cde47b1910f7c995ecf3387"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "8040514fa5e9bd85c55a740473a12706"
server_key_iv: "1832ed0d29c648f1d7b1d726f7313981"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.973750 15723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "91f236bd7cde47b1910f7c995ecf3387"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "8040514fa5e9bd85c55a740473a12706"
server_key_iv: "1832ed0d29c648f1d7b1d726f7313981"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.974892 15723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:16.975634 15738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.975792 15723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.975870 15723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "91f236bd7cde47b1910f7c995ecf3387"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "8040514fa5e9bd85c55a740473a12706"
server_key_iv: "1832ed0d29c648f1d7b1d726f7313981"
server_key_version: "encryptionkey@0"
I20260923 22:10:16.975919 15723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:16.984433 15723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:16.985980 15723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:16.986070 15723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:16.986217 15723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:16.986490 15723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:16.986518 15723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.986536 15723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:16.986577 15723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:16.991950 15723 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:45651
I20260923 22:10:16.992013 15851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:45651 every 8 connection(s)
I20260923 22:10:16.992292 15723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:16.993423 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15723
I20260923 22:10:16.993518 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:16.993662 1670 external_mini_cluster.cc:1428] Setting key aa6a7b658fc397afef705e2e598b0d2c
Sep 23 22:10:16 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201416, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:17.000697 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.992929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:36799 (local address 127.1.161.190:44727)
0923 22:10:16.992983 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:16.992985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:16.993123 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:10:16.993560 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:16.993562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:16.993573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:16.993619 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:16.994599 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.995090 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.996031 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.996253 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.996329 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.998140 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:16.998141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:16.998142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:16.998149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:16.999506 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:16.999847 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:16.999848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:16.999848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:16.999898 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.000123 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.000123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.000124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.000221 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.000289 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.000513 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.000580 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:10:17.000867 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:16.992886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:36799)
0923 22:10:16.993006 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:16.993042 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:16.993463 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:16.993672 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:16.993675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:16.994300 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:16.994520 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.994524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.995194 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.995195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.995945 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:16.995947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:16.996339 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:16.996339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:16.996456 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:16.996893 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:16.996899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:16.998050 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:16.999580 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.999586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.999588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:16.999778 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:16.999973 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:16.999974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:16.999974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.000060 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.000281 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.000286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.000398 (+ 112us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.000483 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.000628 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":55,"threads_started":1}
I20260923 22:10:17.001302 15852 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:17.001369 15852 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:17.001545 15852 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:17.001888 15393 ts_manager.cc:194] Registered new tserver with Master: 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651)
I20260923 22:10:17.002226 15393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:36799
I20260923 22:10:17.005240 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:17.011082 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.006473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.1.161.190:44727)
0923 22:10:17.006504 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.006506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.006648 (+ 142us) server_negotiation.cc:407] Connection header received
0923 22:10:17.006693 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.006694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.006700 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.006745 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.007241 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.007629 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.008251 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.008391 (+ 140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.008430 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.009328 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.009329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.009329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.009333 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.010254 (+ 921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.010514 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.010514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.010515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.010553 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.010705 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.010705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.010705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.010811 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.010877 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.010962 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.011011 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:10:17.012315 15393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:17.013612 15393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:17.019260 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37407 (local address 127.0.0.1:36524)
0923 22:10:17.015772 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.015824 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.015879 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.016030 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.016031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.016033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.016067 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.016068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.017267 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.017267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.018024 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.018026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.018946 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.018947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.019023 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.019035 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.019103 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.019148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":49,"threads_started":1}
I20260923 22:10:17.019287 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.1.161.129:37407)
0923 22:10:17.015892 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.015896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.015908 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:10:17.015943 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.015945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.015957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.016028 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.016132 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.017161 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.018133 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.018863 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.018936 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.018985 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.019129 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.019190 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":86,"threads_started":1}
I20260923 22:10:17.020087 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:45651 (local address 127.0.0.1:38028)
0923 22:10:17.016345 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.016353 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.016394 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.016617 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.016618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.016620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.016652 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.016653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.017663 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.017664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.018668 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.018671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.019836 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.019837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.019895 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.019899 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.019962 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.019997 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":46,"threads_started":1}
I20260923 22:10:17.020094 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.1.161.131:45651)
0923 22:10:17.015875 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.015880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.016459 (+ 579us) server_negotiation.cc:407] Connection header received
0923 22:10:17.016491 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.016493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.016505 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.016565 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.016723 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.017571 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.018751 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.019750 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.019819 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.019865 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.019976 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.020011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260923 22:10:17.019964 15518 tablet_service.cc:1467] Processing CreateTablet for tablet bcb537c9b59347be87dc41f1981e63ce (DEFAULT_TABLE table=test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.020324 15518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb537c9b59347be87dc41f1981e63ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.020653 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35431 (local address 127.0.0.1:43814)
0923 22:10:17.015784 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.015819 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.015878 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.016249 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.016250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.016252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.016288 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.016289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.018153 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.018153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.019219 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.019222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.020437 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.020438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.020503 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.020507 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.020558 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.020593 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":68,"threads_started":1}
I20260923 22:10:17.020686 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.015600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.1.161.130:35431)
0923 22:10:17.016120 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.016123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.016129 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:10:17.016153 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.016154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.016170 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.016240 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.016357 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.017171 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.019543 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.020332 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.020424 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.020483 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.020575 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.020612 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":89,"threads_started":1}
I20260923 22:10:17.022701 15866 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap starting.
I20260923 22:10:17.023355 15866 tablet_bootstrap.cc:654] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.023232 15652 tablet_service.cc:1467] Processing CreateTablet for tablet bcb537c9b59347be87dc41f1981e63ce (DEFAULT_TABLE table=test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.023313 15786 tablet_service.cc:1467] Processing CreateTablet for tablet bcb537c9b59347be87dc41f1981e63ce (DEFAULT_TABLE table=test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.023564 15652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb537c9b59347be87dc41f1981e63ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.023614 15786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb537c9b59347be87dc41f1981e63ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.023888 15866 log.cc:824] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.024602 15866 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: No bootstrap required, opened a new log
I20260923 22:10:17.024668 15866 ts_tablet_manager.cc:1397] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:17.025966 15868 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Bootstrap starting.
I20260923 22:10:17.026487 15869 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Bootstrap starting.
I20260923 22:10:17.026485 15866 raft_consensus.cc:348] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.026654 15866 raft_consensus.cc:374] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.026707 15866 raft_consensus.cc:729] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbeeba9b7f4eca9308fa1733d1bab9, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.026618 15868 tablet_bootstrap.cc:654] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.026815 15866 consensus_queue.cc:260] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.027063 15866 ts_tablet_manager.cc:1428] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:10:17.027098 15869 tablet_bootstrap.cc:654] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.027228 15584 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
I20260923 22:10:17.027374 15868 log.cc:824] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.027908 15869 log.cc:824] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.028154 15868 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: No bootstrap required, opened a new log
I20260923 22:10:17.028216 15868 ts_tablet_manager.cc:1397] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:17.028610 15869 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: No bootstrap required, opened a new log
I20260923 22:10:17.028681 15869 ts_tablet_manager.cc:1397] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:17.029452 15868 raft_consensus.cc:348] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.029588 15868 raft_consensus.cc:374] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.029644 15868 raft_consensus.cc:729] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931c70f66584cbfb005c5b9b1d1ac26, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.029744 15868 consensus_queue.cc:260] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.030015 15868 ts_tablet_manager.cc:1428] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:17.030053 15718 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
I20260923 22:10:17.029953 15869 raft_consensus.cc:348] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.030102 15869 raft_consensus.cc:374] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.030140 15869 raft_consensus.cc:729] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f236bd7cde47b1910f7c995ecf3387, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.030229 15869 consensus_queue.cc:260] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.030470 15869 ts_tablet_manager.cc:1428] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:17.030496 15852 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
W20260923 22:10:17.113838 15719 tablet.cc:2369] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:17.116287 15870 raft_consensus.cc:482] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:17.116369 15870 raft_consensus.cc:504] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.116678 15870 leader_election.cc:288] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431), 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651)
I20260923 22:10:17.119339 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.116844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35431 (local address 127.1.161.129:43519)
0923 22:10:17.116892 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.116902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.116941 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.117141 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.117142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.117144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.117234 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.117236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.117846 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.117847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.118551 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.118553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.119054 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.119055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.119125 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.119133 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.119239 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.119273 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 22:10:17.119338 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.116921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:43519 (local address 127.1.161.130:35431)
0923 22:10:17.116970 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.116972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.116981 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:17.117007 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.117008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.117017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.117099 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.117302 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.117775 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.118643 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.118980 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.119055 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.119063 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.119249 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.119282 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:10:17.119449 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.117041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:37955 (local address 127.1.161.131:45651)
0923 22:10:17.117104 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.117106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.117169 (+ 63us) server_negotiation.cc:407] Connection header received
0923 22:10:17.117232 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.117232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.117246 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.117300 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.117436 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.117834 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.118765 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.119095 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.119183 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.119203 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.119342 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.119377 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:10:17.119596 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.116967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:45651 (local address 127.1.161.129:37955)
0923 22:10:17.117108 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.117115 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.117174 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.117302 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.117303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.117304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.117372 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.117373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.117912 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.117913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.118670 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.118672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.119198 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.119199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.119259 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.119262 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.119324 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.119358 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260923 22:10:17.119643 15672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb537c9b59347be87dc41f1981e63ce" candidate_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" is_pre_election: true
I20260923 22:10:17.119776 15672 raft_consensus.cc:2454] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6bbeeba9b7f4eca9308fa1733d1bab9 in term 0.
I20260923 22:10:17.119875 15806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb537c9b59347be87dc41f1981e63ce" candidate_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f236bd7cde47b1910f7c995ecf3387" is_pre_election: true
I20260923 22:10:17.119987 15472 leader_election.cc:302] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9, c931c70f66584cbfb005c5b9b1d1ac26; no voters:
I20260923 22:10:17.120023 15806 raft_consensus.cc:2454] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6bbeeba9b7f4eca9308fa1733d1bab9 in term 0.
I20260923 22:10:17.120088 15870 raft_consensus.cc:2781] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:17.120163 15870 raft_consensus.cc:482] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:17.120187 15870 raft_consensus.cc:3037] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.120833 15870 raft_consensus.cc:504] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.120954 15870 leader_election.cc:288] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 1 election: Requested vote from peers c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431), 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651)
I20260923 22:10:17.121117 15672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb537c9b59347be87dc41f1981e63ce" candidate_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
I20260923 22:10:17.121142 15806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb537c9b59347be87dc41f1981e63ce" candidate_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f236bd7cde47b1910f7c995ecf3387"
I20260923 22:10:17.121197 15672 raft_consensus.cc:3037] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.121210 15806 raft_consensus.cc:3037] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.122083 15672 raft_consensus.cc:2454] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6bbeeba9b7f4eca9308fa1733d1bab9 in term 1.
I20260923 22:10:17.122082 15806 raft_consensus.cc:2454] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6bbeeba9b7f4eca9308fa1733d1bab9 in term 1.
I20260923 22:10:17.122236 15472 leader_election.cc:302] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f236bd7cde47b1910f7c995ecf3387, a6bbeeba9b7f4eca9308fa1733d1bab9; no voters:
I20260923 22:10:17.122325 15870 raft_consensus.cc:2781] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:17.122458 15870 raft_consensus.cc:686] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 LEADER]: Becoming Leader. State: Replica: a6bbeeba9b7f4eca9308fa1733d1bab9, State: Running, Role: LEADER
I20260923 22:10:17.122546 15870 consensus_queue.cc:237] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.123325 15392 catalog_manager.cc:5495] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } health_report { overall_health: UNKNOWN } } }
I20260923 22:10:17.135910 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.133958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.1.161.129:37407)
0923 22:10:17.134005 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.134007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.134014 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:17.134069 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.134069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.134081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.134130 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:17.134314 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.134659 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.135349 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.135493 (+ 144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.135537 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.135686 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:17.135764 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:17.135813 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.135845 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.135857 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:10:17.137308 15806 raft_consensus.cc:1261] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Refusing update from remote peer a6bbeeba9b7f4eca9308fa1733d1bab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:17.137355 15672 raft_consensus.cc:1261] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Refusing update from remote peer a6bbeeba9b7f4eca9308fa1733d1bab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:17.137588 15870 consensus_queue.cc:1035] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:17.137727 15876 consensus_queue.cc:1035] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:17.141701 15393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:17.141772 15393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:17.142481 15393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Sending DeleteTablet for 3 replicas of tablet bcb537c9b59347be87dc41f1981e63ce
I20260923 22:10:17.142765 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15455
I20260923 22:10:17.142769 15518 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb537c9b59347be87dc41f1981e63ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:36524
I20260923 22:10:17.142822 15652 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb537c9b59347be87dc41f1981e63ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260923 22:10:17.142966 15786 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb537c9b59347be87dc41f1981e63ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38028
I20260923 22:10:17.143518 15890 tablet_replica.cc:330] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: stopping tablet replica
I20260923 22:10:17.143599 15890 raft_consensus.cc:2229] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:17.143716 15890 raft_consensus.cc:2258] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.143834 15889 tablet_replica.cc:330] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: stopping tablet replica
I20260923 22:10:17.143908 15889 raft_consensus.cc:2229] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:17.143991 15889 raft_consensus.cc:2258] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.144109 15890 ts_tablet_manager.cc:1905] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:17.144380 15889 ts_tablet_manager.cc:1905] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:17.145483 15890 ts_tablet_manager.cc:1918] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:17.145524 15890 log.cc:1196] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/bcb537c9b59347be87dc41f1981e63ce
I20260923 22:10:17.145771 15890 ts_tablet_manager.cc:1939] T bcb537c9b59347be87dc41f1981e63ce P 91f236bd7cde47b1910f7c995ecf3387: Deleting consensus metadata
I20260923 22:10:17.146112 15378 catalog_manager.cc:4841] TS 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): tablet bcb537c9b59347be87dc41f1981e63ce (table test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]) successfully deleted
I20260923 22:10:17.146183 15889 ts_tablet_manager.cc:1918] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:17.146253 15889 log.cc:1196] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/bcb537c9b59347be87dc41f1981e63ce
I20260923 22:10:17.146543 15889 ts_tablet_manager.cc:1939] T bcb537c9b59347be87dc41f1981e63ce P c931c70f66584cbfb005c5b9b1d1ac26: Deleting consensus metadata
I20260923 22:10:17.146881 15378 catalog_manager.cc:4841] TS c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): tablet bcb537c9b59347be87dc41f1981e63ce (table test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]) successfully deleted
W20260923 22:10:17.148934 15379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:37407 (error 108)
I20260923 22:10:17.149149 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15589
I20260923 22:10:17.149326 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.149173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37407 (local address 127.0.0.1:36536)
0923 22:10:17.149238 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.149293 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 22:10:17.150178 15379 catalog_manager.cc:4584] TS a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcb537c9b59347be87dc41f1981e63ce: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37407: connect: Connection refused (error 111)
I20260923 22:10:17.153245 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15723
I20260923 22:10:17.157824 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15361
I20260923 22:10:17.162874 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--webserver_interface=127.1.161.190
--webserver_port=39241
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:17.235914 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:17.236102 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:17.236135 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:17.237151 15892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:17.237200 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:17.237219 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:17.237237 15892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:17.237259 15892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:17.238510 15892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=39241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:17.238739 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:17.238926 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:17.246313 15901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.246317 15899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.246320 15898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.246619 15892 server_base.cc:1034] running on GCE node
I20260923 22:10:17.246858 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:17.247159 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:17.248324 15892 hybrid_clock.cc:648] HybridClock initialized: now 1790201417248319 us; error 47 us; skew 500 ppm
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:17.250341 15892 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:17.250808 15892 webserver.cc:460] Webserver started at http://127.1.161.190:39241/ using document root <none> and password file <none>
I20260923 22:10:17.251024 15892 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:17.251101 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:17.252408 15892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:17.253015 15908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.253168 15892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:17.253255 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "65759b0ed96e49328c4d4b690a74829d"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "ed32da83ea4c18e8b8c059c9066a4829"
server_key_iv: "0c657f6c3fea18e7e5bc943ceb2a7289"
server_key_version: "encryptionkey@0"
I20260923 22:10:17.253582 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:17.262324 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:17.263808 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:17.263937 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:17.267445 15892 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:44727
I20260923 22:10:17.267470 15960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:44727 every 8 connection(s)
I20260923 22:10:17.267803 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:17.269008 15961 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:17.269513 15961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap starting.
I20260923 22:10:17.276790 15961 log.cc:824] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.277480 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15892
I20260923 22:10:17.277712 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:37407
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=45625
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:17.278519 15961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:17.278788 15961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap complete.
I20260923 22:10:17.280885 15961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:17.281112 15961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.281306 15961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:17.281396 15961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:17.281452 15961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:17.281499 15961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:17.282363 15961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:17.282462 15961 leader_election.cc:302] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65759b0ed96e49328c4d4b690a74829d; no voters:
I20260923 22:10:17.282629 15961 leader_election.cc:288] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:17.282665 15965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:17.282775 15965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 LEADER]: Becoming Leader. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Running, Role: LEADER
I20260923 22:10:17.282847 15961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:17.282850 15965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:17.283100 15967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:17.283215 15967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:17.283097 15965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65759b0ed96e49328c4d4b690a74829d. Latest consensus state: current_term: 2 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:17.283383 15965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:17.283478 15971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:17.284139 15971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:17.284231 15971 catalog_manager.cc:1259] Loaded cluster ID: e0f4b566caeb4d2f8beca683e906aacb
I20260923 22:10:17.284263 15971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:17.284451 15971 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 22:10:17.288686 15971 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 22:10:17.290948 15971 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:17.291129 15971 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Loaded TSK: 0
I20260923 22:10:17.291359 15971 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:17.350800 15964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:17.351013 15964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:17.351049 15964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:17.352188 15964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:17.352253 15964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:17.352294 15964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:17.353514 15964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:37407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=45625
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:17.353726 15964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:17.353937 15964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:17.358320 15989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.358366 15988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.358419 15991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.358773 15964 server_base.cc:1034] running on GCE node
I20260923 22:10:17.358984 15964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:17.359292 15964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:17.360448 15964 hybrid_clock.cc:648] HybridClock initialized: now 1790201417360434 us; error 33 us; skew 500 ppm
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:17.362977 15964 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:17.363492 15964 webserver.cc:460] Webserver started at http://127.1.161.129:45625/ using document root <none> and password file <none>
I20260923 22:10:17.363758 15964 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:17.363834 15964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:17.365387 15964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:17.366007 15998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.366195 15964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.366279 15964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "3b31d999354c9e785f6a2b9884dc1325"
server_key_iv: "4f019ff5626a1016eec4c260d8c4110a"
server_key_version: "encryptionkey@0"
I20260923 22:10:17.366616 15964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:17.381397 15964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:17.383632 15964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:17.383783 15964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:17.384001 15964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:17.384440 16005 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:17.385619 15964 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:17.385679 15964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:17.385707 15964 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:17.386396 15964 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:17.386437 15964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:17.386492 16005 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap starting.
I20260923 22:10:17.392807 15964 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37407
I20260923 22:10:17.393169 15964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:17.395383 16112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37407 every 8 connection(s)
I20260923 22:10:17.396235 16005 log.cc:824] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.398459 16005 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:17.398721 16005 tablet_bootstrap.cc:492] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap complete.
I20260923 22:10:17.399819 16005 ts_tablet_manager.cc:1397] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:17.400915 16005 raft_consensus.cc:348] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.401212 16005 raft_consensus.cc:729] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbeeba9b7f4eca9308fa1733d1bab9, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.401332 16005 consensus_queue.cc:260] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } }
I20260923 22:10:17.401568 16005 ts_tablet_manager.cc:1428] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:10:17.402125 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 15964
I20260923 22:10:17.402412 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:35431
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=39799
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:17.405254 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.393808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:43153)
0923 22:10:17.395309 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.395358 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.396003 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.396439 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.396442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.396736 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:17.396895 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.396899 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.397987 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.397988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.398807 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.398809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.399409 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.399410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.399542 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.400054 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:17.400067 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:17.401415 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:17.403553 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.403558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.403560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.403819 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.404094 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.404095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.404095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.404240 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.404650 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.404655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.404783 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.404882 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.405086 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"thread_start_us":91,"threads_started":1}
I20260923 22:10:17.405288 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.393868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:43153 (local address 127.1.161.190:44727)
0923 22:10:17.395628 (+ 1760us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.395646 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.395681 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:10:17.396214 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.396220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.396278 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.396397 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.397008 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.397839 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.398890 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.399304 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.399419 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.401526 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.401530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.401530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.401540 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.403468 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.403903 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.403906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.403907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.404015 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.404314 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.404315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.404316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.404564 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.404642 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.404950 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.405082 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1715,"thread_start_us":89,"threads_started":1}
I20260923 22:10:17.406360 16113 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:17.406442 16113 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:17.406600 16113 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:17.407279 15925 ts_manager.cc:194] Registered new tserver with Master: a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407)
I20260923 22:10:17.408282 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:43153
I20260923 22:10:17.409155 16113 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
I20260923 22:10:17.413446 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.407824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37407 (local address 127.0.0.1:36544)
0923 22:10:17.408199 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.408243 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.408309 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.409717 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.409719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.409722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.409777 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.409779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.410947 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.410948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.412393 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.412396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.413194 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.413195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.413281 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.413291 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.413343 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.413384 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":64,"threads_started":1}
I20260923 22:10:17.413715 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.408005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.1.161.129:37407)
0923 22:10:17.408139 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.408143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.408410 (+ 267us) server_negotiation.cc:407] Connection header received
0923 22:10:17.409456 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.409459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.409474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.409541 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.409853 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.410848 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.412485 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.413067 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.413125 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.413165 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.413633 (+ 468us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.413667 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":53,"threads_started":1}
I20260923 22:10:17.414073 16029 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb537c9b59347be87dc41f1981e63ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:36544
I20260923 22:10:17.414611 16122 tablet_replica.cc:330] stopping tablet replica
I20260923 22:10:17.414700 16122 raft_consensus.cc:2229] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:17.414778 16122 raft_consensus.cc:2258] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.415007 16122 ts_tablet_manager.cc:1905] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:17.416961 16122 ts_tablet_manager.cc:1918] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:17.417018 16122 log.cc:1196] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/bcb537c9b59347be87dc41f1981e63ce
I20260923 22:10:17.417272 16122 ts_tablet_manager.cc:1939] T bcb537c9b59347be87dc41f1981e63ce P a6bbeeba9b7f4eca9308fa1733d1bab9: Deleting consensus metadata
I20260923 22:10:17.417788 15911 catalog_manager.cc:4841] TS a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407): tablet bcb537c9b59347be87dc41f1981e63ce (table test-table [id=fd03ee2173ab4c15927b5cdb26e2cfbc]) successfully deleted
W20260923 22:10:17.477159 16119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:17.477312 16119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:17.477331 16119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:17.478348 16119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:17.478379 16119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:17.478403 16119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:17.479653 16119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:35431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=39799
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:17.479807 16119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:17.479944 16119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:17.484381 16127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.484376 16130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.484391 16119 server_base.cc:1034] running on GCE node
W20260923 22:10:17.484366 16128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.484779 16119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:17.485018 16119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:17.486168 16119 hybrid_clock.cc:648] HybridClock initialized: now 1790201417486157 us; error 23 us; skew 500 ppm
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:17.488368 16119 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:17.488844 16119 webserver.cc:460] Webserver started at http://127.1.161.130:39799/ using document root <none> and password file <none>
I20260923 22:10:17.489040 16119 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:17.489111 16119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:17.490221 16119 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:17.490782 16137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.490953 16119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:17.491045 16119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "60b29f5163f41e4a95f15e4b21e44f10"
server_key_iv: "c4d8dae3bc782f8d0de1c8cfb27541b6"
server_key_version: "encryptionkey@0"
I20260923 22:10:17.491405 16119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:17.505185 16119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:17.506846 16119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:17.506987 16119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:17.507217 16119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:17.507468 16119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:17.507514 16119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.507543 16119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:17.507566 16119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.512655 16119 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:35431
I20260923 22:10:17.512689 16250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:35431 every 8 connection(s)
I20260923 22:10:17.512954 16119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:17.516966 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16119
I20260923 22:10:17.517212 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:45651
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=39411
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:17.520844 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.513485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:51021 (local address 127.1.161.190:44727)
0923 22:10:17.513540 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.513542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.513737 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:10:17.514177 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.514178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.514187 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.514230 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.515094 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.515567 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.516347 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.516506 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.516547 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.518437 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.518438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.518438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.518444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.519617 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.519983 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.519984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.519985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.520033 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.520256 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.520256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.520257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.520348 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.520407 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.520634 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.520757 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:10:17.521026 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.513478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:51021)
0923 22:10:17.513624 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.513656 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.514090 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.514292 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.514295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.514869 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:17.515018 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.515022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.515633 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.515633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.516282 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.516284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.516565 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.516566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.516645 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.517080 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:17.517088 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:17.518349 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:17.519716 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.519722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.519724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.519906 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.520090 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.520091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.520091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.520187 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.520411 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.520416 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.520526 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.520615 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.520779 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260923 22:10:17.521574 16251 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:17.521667 16251 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:17.521823 16251 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:17.522177 15925 ts_manager.cc:194] Registered new tserver with Master: c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
I20260923 22:10:17.522644 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:51021
W20260923 22:10:17.590411 16254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:17.590570 16254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:17.590590 16254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:17.591818 16254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:17.591871 16254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:17.591915 16254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:17.593102 16254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:45651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=39411
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:17.593261 16254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:17.593484 16254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:17.597992 16260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.598001 16262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.598001 16259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.598117 16254 server_base.cc:1034] running on GCE node
I20260923 22:10:17.598433 16254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:17.598649 16254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:17.599798 16254 hybrid_clock.cc:648] HybridClock initialized: now 1790201417599795 us; error 33 us; skew 500 ppm
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:17.601972 16254 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:17.602284 16254 webserver.cc:460] Webserver started at http://127.1.161.131:39411/ using document root <none> and password file <none>
I20260923 22:10:17.602432 16254 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:17.602476 16254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:17.603658 16254 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:17.604130 16269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.604311 16254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.604370 16254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "91f236bd7cde47b1910f7c995ecf3387"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "8040514fa5e9bd85c55a740473a12706"
server_key_iv: "1832ed0d29c648f1d7b1d726f7313981"
server_key_version: "encryptionkey@0"
I20260923 22:10:17.604604 16254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:17.613493 16254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:17.614928 16254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:17.615016 16254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:17.615226 16254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:17.615444 16254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:17.615471 16254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.615489 16254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:17.615500 16254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.620782 16254 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:45651
I20260923 22:10:17.620833 16382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:45651 every 8 connection(s)
I20260923 22:10:17.621137 16254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:17.621687 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16254
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:17.629012 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.622033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.1.161.190:44727)
0923 22:10:17.622148 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.622150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.622647 (+ 497us) server_negotiation.cc:407] Connection header received
0923 22:10:17.622718 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.622719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.622726 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.622768 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.623447 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.624102 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.625044 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.625224 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.625277 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.626631 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.626632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.626632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.626637 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.627974 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.628270 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.628271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.628272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.628322 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.628499 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.628500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.628500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.628593 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.628775 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.628808 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.628903 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260923 22:10:17.630136 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.621823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:60065 (local address 127.1.161.190:44727)
0923 22:10:17.621901 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.621903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.622059 (+ 156us) server_negotiation.cc:407] Connection header received
0923 22:10:17.622491 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.622492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.622500 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.622545 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:17.623590 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.624136 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.625235 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.625443 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.625482 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.627680 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:17.627681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:17.627681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:17.627686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:17.628872 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.629335 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.629336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.629337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.629388 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:17.629611 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:17.629612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:17.629612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:17.629711 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:17.629772 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.629999 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.630066 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:10:17.630280 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.621793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:60065)
0923 22:10:17.621965 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.621996 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.622412 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.622635 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.622639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.623312 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:17.623529 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.623534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.624215 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.624216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.625176 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.625178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.625615 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.625616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.625726 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.626093 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:17.626101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:17.627589 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:17.628965 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.628971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.628973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.629268 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.629451 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:17.629452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:17.629452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:17.629546 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:17.629766 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:17.629772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:17.629886 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.629970 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.630090 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":108,"threads_started":1}
I20260923 22:10:17.630668 16383 heartbeater.cc:347] Connected to a master server at 127.1.161.190:44727
I20260923 22:10:17.630735 16383 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:17.630900 16383 heartbeater.cc:511] Master 127.1.161.190:44727 requested a full tablet report, sending...
I20260923 22:10:17.631263 15925 ts_manager.cc:194] Registered new tserver with Master: 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651)
I20260923 22:10:17.631608 15925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:60065
I20260923 22:10:17.640259 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:17.642460 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.640581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.1.161.190:44727)
0923 22:10:17.640615 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.640617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.640721 (+ 104us) server_negotiation.cc:407] Connection header received
0923 22:10:17.640761 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.640762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.640785 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.640839 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:17.640931 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.641348 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.642008 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.642145 (+ 137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.642184 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.642284 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:17.642346 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:17.642389 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.642416 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.642427 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:10:17.642660 15925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:17.644038 15925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:17.646096 16029 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f (DEFAULT_TABLE table=test-table [id=a795481624e7476eba643224cfc8a038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.646273 16029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec0b10f9f0448a88c85964b7fa26d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.647559 16005 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap starting.
I20260923 22:10:17.647881 16005 tablet_bootstrap.cc:654] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.648411 16005 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: No bootstrap required, opened a new log
I20260923 22:10:17.648461 16005 ts_tablet_manager.cc:1397] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:17.648583 16005 raft_consensus.cc:348] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.648665 16005 raft_consensus.cc:374] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.648713 16005 raft_consensus.cc:729] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbeeba9b7f4eca9308fa1733d1bab9, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.648759 16005 consensus_queue.cc:260] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.648844 16005 ts_tablet_manager.cc:1428] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.649523 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.645887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35431 (local address 127.0.0.1:43830)
0923 22:10:17.646083 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.646091 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.646161 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.646437 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.646438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.646440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.646477 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.646478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.647465 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.647466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.648459 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.648462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.649327 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.649327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.649382 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.649385 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.649437 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.649468 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260923 22:10:17.649535 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.646063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.1.161.130:35431)
0923 22:10:17.646286 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.646289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.646297 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:17.646347 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.646349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.646366 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.646440 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.646536 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.647339 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.648562 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.649256 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.649312 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.649345 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.649447 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.649476 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260923 22:10:17.650174 16185 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f (DEFAULT_TABLE table=test-table [id=a795481624e7476eba643224cfc8a038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.650471 16185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec0b10f9f0448a88c85964b7fa26d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.650679 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.645777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:45651 (local address 127.0.0.1:38032)
0923 22:10:17.645812 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.645823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.645872 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.647748 (+ 1876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.647749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.647751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.647784 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.647785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.648749 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.648750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.649586 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.649588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.650450 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.650451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.650493 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.650495 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.650567 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.650608 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":2}
I20260923 22:10:17.650700 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.645859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.1.161.131:45651)
0923 22:10:17.647608 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.647611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.647620 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:17.647651 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.647652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.647665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.647737 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.647830 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.648630 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.649798 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.650372 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.650431 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.650475 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.650579 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.650615 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1717,"thread_start_us":80,"threads_started":1}
I20260923 22:10:17.651386 16317 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f (DEFAULT_TABLE table=test-table [id=a795481624e7476eba643224cfc8a038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:17.651675 16317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec0b10f9f0448a88c85964b7fa26d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:17.653147 16393 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Bootstrap starting.
I20260923 22:10:17.653741 16393 tablet_bootstrap.cc:654] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.654281 16393 log.cc:824] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.654474 16394 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Bootstrap starting.
I20260923 22:10:17.654995 16393 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: No bootstrap required, opened a new log
I20260923 22:10:17.655069 16393 ts_tablet_manager.cc:1397] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:17.655076 16394 tablet_bootstrap.cc:654] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:17.655619 16394 log.cc:824] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:17.656268 16394 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: No bootstrap required, opened a new log
I20260923 22:10:17.656335 16394 ts_tablet_manager.cc:1397] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:10:17.656986 16393 raft_consensus.cc:348] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.657158 16393 raft_consensus.cc:374] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.657205 16393 raft_consensus.cc:729] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931c70f66584cbfb005c5b9b1d1ac26, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.657308 16393 consensus_queue.cc:260] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.657552 16393 ts_tablet_manager.cc:1428] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:10:17.657672 16251 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
I20260923 22:10:17.658176 16394 raft_consensus.cc:348] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.658362 16394 raft_consensus.cc:374] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:17.658420 16394 raft_consensus.cc:729] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f236bd7cde47b1910f7c995ecf3387, State: Initialized, Role: FOLLOWER
I20260923 22:10:17.658581 16394 consensus_queue.cc:260] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.658897 16394 ts_tablet_manager.cc:1428] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:10:17.658916 16383 heartbeater.cc:503] Master 127.1.161.190:44727 was elected leader, sending a full tablet report...
W20260923 22:10:17.763477 16252 tablet.cc:2369] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:17.859309 16398 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:17.859424 16398 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.859717 16398 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431), a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407)
I20260923 22:10:17.862323 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.859971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:35325 (local address 127.1.161.130:35431)
0923 22:10:17.860017 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.860019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.860042 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:10:17.860066 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.860066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.860076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.860134 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.860277 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.860717 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.861646 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.862011 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.862085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.862092 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.862233 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.862266 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:10:17.862323 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.859874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35431 (local address 127.1.161.131:35325)
0923 22:10:17.859950 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.859962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.860011 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.860145 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.860146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.860148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.860213 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.860215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.860799 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.860800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.861552 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.861554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.862083 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.862083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.862158 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.862160 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.862226 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.862259 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":20}
I20260923 22:10:17.862394 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.859975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:33565 (local address 127.1.161.129:37407)
0923 22:10:17.860015 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.860017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.860112 (+ 95us) server_negotiation.cc:407] Connection header received
0923 22:10:17.860172 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.860173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.860191 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.860254 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:17.860395 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.860827 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.861744 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.862084 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.862159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.862165 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.862299 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.862332 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:10:17.862421 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.859883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37407 (local address 127.1.161.131:33565)
0923 22:10:17.860047 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.860056 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:17.860125 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:17.860258 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:17.860259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:17.860260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:17.860331 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.860332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.860912 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.860913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.861637 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:17.861639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.862159 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.862160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:17.862226 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.862229 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:17.862293 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:10:17.862326 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260923 22:10:17.862618 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec0b10f9f0448a88c85964b7fa26d1f" candidate_uuid: "91f236bd7cde47b1910f7c995ecf3387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" is_pre_election: true
I20260923 22:10:17.862730 16205 raft_consensus.cc:2454] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f236bd7cde47b1910f7c995ecf3387 in term 0.
I20260923 22:10:17.862927 16271 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26; no voters:
I20260923 22:10:17.863029 16398 raft_consensus.cc:2781] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:17.863078 16398 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:17.863092 16398 raft_consensus.cc:3037] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.863195 16053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec0b10f9f0448a88c85964b7fa26d1f" candidate_uuid: "91f236bd7cde47b1910f7c995ecf3387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" is_pre_election: true
I20260923 22:10:17.863354 16053 raft_consensus.cc:2454] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f236bd7cde47b1910f7c995ecf3387 in term 0.
I20260923 22:10:17.863890 16398 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.864019 16398 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [CANDIDATE]: Term 1 election: Requested vote from peers c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431), a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407)
I20260923 22:10:17.864151 16053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec0b10f9f0448a88c85964b7fa26d1f" candidate_uuid: "91f236bd7cde47b1910f7c995ecf3387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
I20260923 22:10:17.864169 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec0b10f9f0448a88c85964b7fa26d1f" candidate_uuid: "91f236bd7cde47b1910f7c995ecf3387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
I20260923 22:10:17.864224 16053 raft_consensus.cc:3037] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.864231 16205 raft_consensus.cc:3037] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:17.865015 16205 raft_consensus.cc:2454] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f236bd7cde47b1910f7c995ecf3387 in term 1.
I20260923 22:10:17.865015 16053 raft_consensus.cc:2454] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f236bd7cde47b1910f7c995ecf3387 in term 1.
I20260923 22:10:17.865183 16272 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f236bd7cde47b1910f7c995ecf3387, a6bbeeba9b7f4eca9308fa1733d1bab9; no voters:
I20260923 22:10:17.865281 16398 raft_consensus.cc:2781] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:17.865387 16398 raft_consensus.cc:686] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 1 LEADER]: Becoming Leader. State: Replica: 91f236bd7cde47b1910f7c995ecf3387, State: Running, Role: LEADER
I20260923 22:10:17.865563 16398 consensus_queue.cc:237] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:17.866358 15925 catalog_manager.cc:5495] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "91f236bd7cde47b1910f7c995ecf3387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } health_report { overall_health: UNKNOWN } } }
W20260923 22:10:17.871620 16384 tablet.cc:2369] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:17.873310 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.871534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.1.161.131:45651)
0923 22:10:17.871590 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:17.871592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:17.871600 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:17.871629 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:17.871630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:17.871646 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:17.871718 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:17.871860 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.872242 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.872785 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:17.872926 (+ 141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:17.872965 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:17.873069 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:17.873139 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:17.873193 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:17.873203 (+ 10us) server_negotiation.cc:299] Negotiation successful
0923 22:10:17.873215 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:10:17.874676 16205 raft_consensus.cc:1261] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Refusing update from remote peer 91f236bd7cde47b1910f7c995ecf3387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:17.874687 16053 raft_consensus.cc:1261] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Refusing update from remote peer 91f236bd7cde47b1910f7c995ecf3387: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:17.874998 16398 consensus_queue.cc:1035] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:17.875144 16400 consensus_queue.cc:1035] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:17.877043 16404 mvcc.cc:205] Tried to move back new op lower bound from 7332665007612448768 to 7332665007578181632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665007612448768}]
I20260923 22:10:17.878772 15925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:17.878832 15925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:17.879441 15925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Sending DeleteTablet for 3 replicas of tablet 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:17.879671 16317 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38032
I20260923 22:10:17.879726 16185 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:43830
I20260923 22:10:17.879877 16413 tablet_replica.cc:330] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: stopping tablet replica
I20260923 22:10:17.879881 16029 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec0b10f9f0448a88c85964b7fa26d1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:17 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:36544
I20260923 22:10:17.879942 16122 tablet_replica.cc:330] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: stopping tablet replica
I20260923 22:10:17.879967 16413 raft_consensus.cc:2229] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:17.879983 16122 raft_consensus.cc:2229] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:17.880013 16122 raft_consensus.cc:2258] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.880043 16413 raft_consensus.cc:2258] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.880100 16414 tablet_replica.cc:330] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: stopping tablet replica
I20260923 22:10:17.880231 16414 raft_consensus.cc:2229] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:17.880290 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15964
I20260923 22:10:17.880388 16414 raft_consensus.cc:2258] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:17.880411 16413 ts_tablet_manager.cc:1905] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:17.880862 16414 ts_tablet_manager.cc:1905] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:17.882226 16413 ts_tablet_manager.cc:1918] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:17.882293 16413 log.cc:1196] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:17.882521 16414 ts_tablet_manager.cc:1918] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:17.882542 16413 ts_tablet_manager.cc:1939] T 1ec0b10f9f0448a88c85964b7fa26d1f P 91f236bd7cde47b1910f7c995ecf3387: Deleting consensus metadata
I20260923 22:10:17.882668 16414 log.cc:1196] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:17.882952 15910 catalog_manager.cc:4841] TS 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): tablet 1ec0b10f9f0448a88c85964b7fa26d1f (table test-table [id=a795481624e7476eba643224cfc8a038]) successfully deleted
I20260923 22:10:17.883157 16414 ts_tablet_manager.cc:1939] T 1ec0b10f9f0448a88c85964b7fa26d1f P c931c70f66584cbfb005c5b9b1d1ac26: Deleting consensus metadata
I20260923 22:10:17.883587 15910 catalog_manager.cc:4841] TS c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): tablet 1ec0b10f9f0448a88c85964b7fa26d1f (table test-table [id=a795481624e7476eba643224cfc8a038]) successfully deleted
W20260923 22:10:17.886345 15911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:37407 (error 108)
I20260923 22:10:17.886809 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16119
I20260923 22:10:17.886931 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:17.886793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37407 (local address 127.0.0.1:36560)
0923 22:10:17.886855 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:17.886900 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 22:10:17.887320 15911 catalog_manager.cc:4584] TS a6bbeeba9b7f4eca9308fa1733d1bab9 (127.1.161.129:37407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ec0b10f9f0448a88c85964b7fa26d1f: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37407: connect: Connection refused (error 111)
I20260923 22:10:17.892208 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16254
I20260923 22:10:17.898463 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 15892
I20260923 22:10:17.903725 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--webserver_interface=127.1.161.190
--webserver_port=39241
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:17.975054 16416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:17.975308 16416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:17.975337 16416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:17.976406 16416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:17.976436 16416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:17.976449 16416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:17.976460 16416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:17.976471 16416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:17.977653 16416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=39241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:17.977878 16416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:17.978048 16416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:17.982239 16421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.982298 16424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:17.982307 16422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:17.982606 16416 server_base.cc:1034] running on GCE node
I20260923 22:10:17.982836 16416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:17.983093 16416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:17.984241 16416 hybrid_clock.cc:648] HybridClock initialized: now 1790201417984240 us; error 52 us; skew 500 ppm
Sep 23 22:10:17 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:17.986235 16416 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:17.986603 16416 webserver.cc:460] Webserver started at http://127.1.161.190:39241/ using document root <none> and password file <none>
I20260923 22:10:17.986786 16416 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:17.986845 16416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:17.987920 16416 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:17.988480 16431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:17.988631 16416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:17.988715 16416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "65759b0ed96e49328c4d4b690a74829d"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "ed32da83ea4c18e8b8c059c9066a4829"
server_key_iv: "0c657f6c3fea18e7e5bc943ceb2a7289"
server_key_version: "encryptionkey@0"
I20260923 22:10:17.988976 16416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:18.007000 16416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:18.008562 16416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:18.008692 16416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:18.012749 16416 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:44727
I20260923 22:10:18.012775 16483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:44727 every 8 connection(s)
I20260923 22:10:18.013069 16416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:18.014261 16484 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:18.014810 16484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap starting.
I20260923 22:10:18.018448 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16416
I20260923 22:10:18.018705 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:37407
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=45625
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:18.022621 16484 log.cc:824] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:18.024581 16484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:18.024842 16484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d: Bootstrap complete.
I20260923 22:10:18.026643 16484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:18.026852 16484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Initialized, Role: FOLLOWER
I20260923 22:10:18.027010 16484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:18.027061 16484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:18.027117 16484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:18.027184 16484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:10:18.028031 16484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:18.028149 16484 leader_election.cc:302] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65759b0ed96e49328c4d4b690a74829d; no voters:
I20260923 22:10:18.028316 16484 leader_election.cc:288] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:10:18.028344 16488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:10:18.028431 16488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [term 3 LEADER]: Becoming Leader. State: Replica: 65759b0ed96e49328c4d4b690a74829d, State: Running, Role: LEADER
I20260923 22:10:18.028534 16484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:18.028515 16488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } }
I20260923 22:10:18.028854 16489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:18.029037 16489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:18.028987 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65759b0ed96e49328c4d4b690a74829d. Latest consensus state: current_term: 3 leader_uuid: "65759b0ed96e49328c4d4b690a74829d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65759b0ed96e49328c4d4b690a74829d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 44727 } } }
I20260923 22:10:18.029232 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65759b0ed96e49328c4d4b690a74829d [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:18.029345 16498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:18.029966 16498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:18.030049 16498 catalog_manager.cc:1259] Loaded cluster ID: e0f4b566caeb4d2f8beca683e906aacb
I20260923 22:10:18.030079 16498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 22:10:18.030244 16498 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcbacb5b9ad google::LogMessage::Fail() at ??:0
@ 0x7fcbacb5e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fcbacb5b4de google::LogMessage::Flush() at ??:0
@ 0x7fcbacb5ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcbb4a800f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fcbb4a898b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcbaea290d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcbaea1e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcbaf8826db start_thread at ??:0
@ 0x7fcbab65961f clone at ??:0
W20260923 22:10:18.099741 16486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:18.099924 16486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:18.099956 16486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:18.100977 16486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:18.101038 16486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:18.101083 16486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:18.102439 16486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:37407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=45625
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:18.102656 16486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:18.102873 16486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:18.107141 16523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:18.107113 16520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:18.107097 16521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:18.107375 16486 server_base.cc:1034] running on GCE node
I20260923 22:10:18.107548 16486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:18.107767 16486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:18.108924 16486 hybrid_clock.cc:648] HybridClock initialized: now 1790201418108913 us; error 24 us; skew 500 ppm
Sep 23 22:10:18 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:18.111722 16486 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:18.112157 16486 webserver.cc:460] Webserver started at http://127.1.161.129:45625/ using document root <none> and password file <none>
I20260923 22:10:18.112356 16486 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:18.112402 16486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:18.113862 16486 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:18.114557 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.114763 16486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:18.114845 16486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "3b31d999354c9e785f6a2b9884dc1325"
server_key_iv: "4f019ff5626a1016eec4c260d8c4110a"
server_key_version: "encryptionkey@0"
I20260923 22:10:18.115209 16486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:18.129267 16486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:18.131517 16486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:18.131656 16486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:18.131882 16486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:18.132315 16537 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:18.133489 16486 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:18.133556 16486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:18.133589 16486 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:18.134311 16486 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:18.134351 16486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:18.134419 16537 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap starting.
I20260923 22:10:18.140542 16486 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37407
I20260923 22:10:18.140794 16644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37407 every 8 connection(s)
I20260923 22:10:18.140951 16486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:18.143256 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16486
I20260923 22:10:18.143498 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:35431
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=39799
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:18.143949 16537 log.cc:824] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:18.146049 16537 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:18.146296 16537 tablet_bootstrap.cc:492] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Bootstrap complete.
I20260923 22:10:18.147354 16537 ts_tablet_manager.cc:1397] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.010s
I20260923 22:10:18.148850 16537 raft_consensus.cc:348] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:18.149207 16537 raft_consensus.cc:729] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6bbeeba9b7f4eca9308fa1733d1bab9, State: Initialized, Role: FOLLOWER
I20260923 22:10:18.149349 16537 consensus_queue.cc:260] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:18.149610 16537 ts_tablet_manager.cc:1428] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260923 22:10:18.195902 16647 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:10:18.141574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:39249)
0923 22:10:18.141806 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.141869 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:18.142445 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:18.195684 (+ 53239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
W20260923 22:10:18.196158 16645 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:44727 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:44727: Client connection negotiation failed: client connection to 127.1.161.190:44727: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:10:18.196493 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.196381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:58181)
0923 22:10:18.196434 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.196474 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:10:18.196821 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.196722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:48101)
0923 22:10:18.196763 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.196806 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 22:10:18.196913 16645 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:10:18.217259 16649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:18.217432 16649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:18.217463 16649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:18.218516 16649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:18.218571 16649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:18.218618 16649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:18.219915 16649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:35431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=39799
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:18.220132 16649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:18.220315 16649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:18.224608 16656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:18.224618 16655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:18.224701 16649 server_base.cc:1034] running on GCE node
W20260923 22:10:18.224618 16658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:18.224961 16649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:18.225173 16649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:18.226333 16649 hybrid_clock.cc:648] HybridClock initialized: now 1790201418226326 us; error 23 us; skew 500 ppm
Sep 23 22:10:18 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:18.228420 16649 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:18.228755 16649 webserver.cc:460] Webserver started at http://127.1.161.130:39799/ using document root <none> and password file <none>
I20260923 22:10:18.228950 16649 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:18.229007 16649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:18.230111 16649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:18.230661 16665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.230841 16649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:18.230932 16649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "c931c70f66584cbfb005c5b9b1d1ac26"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "60b29f5163f41e4a95f15e4b21e44f10"
server_key_iv: "c4d8dae3bc782f8d0de1c8cfb27541b6"
server_key_version: "encryptionkey@0"
I20260923 22:10:18.231261 16649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:18.248464 16649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:18.249969 16649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:18.250104 16649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:18.250290 16649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:18.250531 16649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:18.250577 16649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.250610 16649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:18.250634 16649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.255604 16649 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:35431
I20260923 22:10:18.255632 16778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:35431 every 8 connection(s)
I20260923 22:10:18.255899 16649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:18.256726 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.256451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:51845)
0923 22:10:18.256567 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.256620 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
W20260923 22:10:18.257057 16779 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:44727 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:44727: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
I20260923 22:10:18.257296 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.257221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:60203)
0923 22:10:18.257259 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.257281 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:10:18.257486 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.257446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:56997)
0923 22:10:18.257463 (+ 17us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.257477 (+ 14us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260923 22:10:18.257565 16779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:18.259281 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16649
I20260923 22:10:18.259510 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:45651
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:44727
--webserver_port=39411
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:18.328109 16782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:18.328259 16782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:18.328280 16782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:18.329200 16782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:18.329229 16782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:18.329252 16782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:18.330454 16782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:45651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=39411
--tserver_master_addrs=127.1.161.190:44727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:18.330598 16782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:18.330772 16782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:18.335136 16787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:18.335178 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:18.335229 16782 server_base.cc:1034] running on GCE node
W20260923 22:10:18.335156 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:18.335592 16782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:18.335776 16782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:18.336922 16782 hybrid_clock.cc:648] HybridClock initialized: now 1790201418336922 us; error 35 us; skew 500 ppm
Sep 23 22:10:18 dist-test-slave-npvd krb5kdc[15348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:18.339264 16782 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:18.339684 16782 webserver.cc:460] Webserver started at http://127.1.161.131:39411/ using document root <none> and password file <none>
I20260923 22:10:18.339900 16782 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:18.339959 16782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:18.341187 16782 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:18.341733 16797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.341898 16782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:18.341995 16782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "91f236bd7cde47b1910f7c995ecf3387"
format_stamp: "Formatted at 2026-09-23 22:10:16 on dist-test-slave-npvd"
server_key: "8040514fa5e9bd85c55a740473a12706"
server_key_iv: "1832ed0d29c648f1d7b1d726f7313981"
server_key_version: "encryptionkey@0"
I20260923 22:10:18.342280 16782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:18.354305 16782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:18.355952 16782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:18.356081 16782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:18.356261 16782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:18.356492 16782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:18.356536 16782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.356570 16782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:18.356592 16782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:18.361972 16782 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:45651
I20260923 22:10:18.362011 16910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:45651 every 8 connection(s)
I20260923 22:10:18.362270 16782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:18.363067 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.362773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:60885)
0923 22:10:18.362907 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.362961 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260923 22:10:18.363335 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16782
W20260923 22:10:18.363487 16911 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:44727 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:44727: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
I20260923 22:10:18.363722 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.363656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:58533)
0923 22:10:18.363684 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.363708 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 22:10:18.364006 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:18.363919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:42311)
0923 22:10:18.363966 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:18.363986 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:10:18.364121 16911 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:10:19.197352 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.197177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:42579)
0923 22:10:19.197274 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.197301 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
I20260923 22:10:19.258021 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.257844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:47655)
0923 22:10:19.257950 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.257977 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260923 22:10:19.364518 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.364363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:36801)
0923 22:10:19.364453 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.364479 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":47,"threads_started":1}
I20260923 22:10:19.634076 16918 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:19.634197 16918 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:19.634512 16918 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651), c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
Sep 23 22:10:19 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.131@KRBTEST.COM
Sep 23 22:10:19 dist-test-slave-npvd krb5kdc[15348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.130@KRBTEST.COM
I20260923 22:10:19.643870 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.634822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:60187 (local address 127.1.161.131:45651)
0923 22:10:19.634941 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:19.634953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:19.634986 (+ 33us) server_negotiation.cc:407] Connection header received
0923 22:10:19.635470 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:19.635472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:19.635514 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:19.635627 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:19.636333 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.637053 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.637987 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.638351 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.638458 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:19.640593 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:19.640596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:19.640596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:19.640605 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:19.642419 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:19.642762 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:19.642765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:19.642765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:19.642845 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:19.643092 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:19.643093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:19.643093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:19.643356 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:19.643430 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:19.643673 (+ 243us) server_negotiation.cc:299] Negotiation successful
0923 22:10:19.643775 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":53,"threads_started":1}
I20260923 22:10:19.643883 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.634690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:45651 (local address 127.1.161.129:60187)
0923 22:10:19.634739 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.634750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:19.634815 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:19.635683 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:19.635686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:19.636055 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:19.636227 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:19.636232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.637146 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.637147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:19.637904 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:19.637905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.638434 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.638435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:19.638532 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:19.639118 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:19.639127 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:19.640496 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:19.642496 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:19.642502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:19.642503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:19.642689 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:19.642910 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:19.642911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:19.642911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:19.643016 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:19.643424 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:19.643429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:19.643533 (+ 104us) client_negotiation.cc:769] Sending connection context
0923 22:10:19.643629 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:10:19.643771 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":14}
I20260923 22:10:19.644141 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.634798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:35431 (local address 127.1.161.129:41233)
0923 22:10:19.634902 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:19.634916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:19.634948 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:19.635743 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:19.635745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:19.636054 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:19.636227 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:19.636232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.637156 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.637157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:19.637905 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:19.637907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.638438 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.638439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:19.638533 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:19.639062 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:19.639069 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:19.640878 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:19.642813 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:19.642814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:19.642815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:19.642958 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:19.643248 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:19.643249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:19.643250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:19.643302 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:19.643629 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:19.643630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:19.643665 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 22:10:19.643721 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:10:19.643825 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":73,"thread_start_us":61,"threads_started":1}
I20260923 22:10:19.644470 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:19.634965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:41233 (local address 127.1.161.130:35431)
0923 22:10:19.635086 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:19.635093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:19.635117 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:10:19.635553 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:19.635555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:19.635613 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:19.635737 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:19.636331 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.637055 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.637991 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:19.638353 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:19.638457 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:19.640994 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:19.640997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:19.640997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:19.641005 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:19.642731 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:19.643046 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:19.643049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:19.643049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:19.643140 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:19.643366 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:19.643367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:19.643367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:19.643559 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:19.643643 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:19.643765 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 22:10:19.643871 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":51,"threads_started":1}
W20260923 22:10:19.644618 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
W20260923 22:10:19.645033 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:19.645071 16532 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9; no voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26
I20260923 22:10:19.645164 16918 raft_consensus.cc:2726] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:20.198005 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.197832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:35799)
0923 22:10:20.197928 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.197957 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
I20260923 22:10:20.258584 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.258419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:41921)
0923 22:10:20.258524 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.258553 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260923 22:10:20.365109 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:20.364902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:34165)
0923 22:10:20.365035 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:20.365072 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260923 22:10:21.198627 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.198426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:40367)
0923 22:10:21.198549 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.198582 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260923 22:10:21.259219 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.259007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:47859)
0923 22:10:21.259114 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.259140 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
I20260923 22:10:21.365877 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:21.365680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:42863)
0923 22:10:21.365818 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:21.365848 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260923 22:10:21.506409 16928 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:21.506486 16928 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:21.506659 16928 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651), c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
W20260923 22:10:21.507071 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
W20260923 22:10:21.507153 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:21.507234 16532 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9; no voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26
I20260923 22:10:21.507308 16928 raft_consensus.cc:2726] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:22.199249 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.199036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:52153)
0923 22:10:22.199143 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.199206 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260923 22:10:22.259824 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.259651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:44631)
0923 22:10:22.259753 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.259794 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260923 22:10:22.366508 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:22.366286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:38887)
0923 22:10:22.366436 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:22.366471 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260923 22:10:23.199855 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.199679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:54971)
0923 22:10:23.199788 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.199817 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":48,"threads_started":1}
I20260923 22:10:23.218430 16933 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:23.218523 16933 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:23.218690 16933 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651), c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
W20260923 22:10:23.219063 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
W20260923 22:10:23.219128 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:23.219141 16532 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9; no voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26
I20260923 22:10:23.219234 16933 raft_consensus.cc:2726] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:23.260408 16934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.260209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:43717)
0923 22:10:23.260332 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.260374 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":44,"threads_started":1}
I20260923 22:10:23.367209 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:23.367007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:58863)
0923 22:10:23.367111 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:23.367140 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260923 22:10:24.200402 16936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.200224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:41709)
0923 22:10:24.200331 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.200359 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260923 22:10:24.260929 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.260768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:53823)
0923 22:10:24.260873 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.260898 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260923 22:10:24.367785 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:24.367602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:57509)
0923 22:10:24.367714 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:24.367747 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260923 22:10:25.201041 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.200817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:36635)
0923 22:10:25.200962 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.200994 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260923 22:10:25.261489 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.261283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:54263)
0923 22:10:25.261413 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.261450 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260923 22:10:25.368480 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:25.368297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:48899)
0923 22:10:25.368419 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:25.368447 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260923 22:10:26.201835 16942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.201619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:35889)
0923 22:10:26.201759 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.201792 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260923 22:10:26.262091 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.261912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:42503)
0923 22:10:26.262027 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.262059 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":51,"threads_started":1}
I20260923 22:10:26.369012 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:26.368847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:35417)
0923 22:10:26.368956 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:26.368982 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260923 22:10:27.202450 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.202241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:44579)
0923 22:10:27.202357 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.202393 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260923 22:10:27.262717 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.262561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:59363)
0923 22:10:27.262653 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.262681 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":49,"threads_started":1}
I20260923 22:10:27.369729 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:27.369516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:52139)
0923 22:10:27.369649 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:27.369679 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260923 22:10:28.200434 16948 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:28.200536 16948 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:28.200740 16948 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651), c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
W20260923 22:10:28.201169 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
W20260923 22:10:28.201234 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:28.201283 16532 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9; no voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26
I20260923 22:10:28.201371 16948 raft_consensus.cc:2726] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:28.202952 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.202815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:58949)
0923 22:10:28.202898 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.202923 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":43,"threads_started":1}
I20260923 22:10:28.263348 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.263114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:56991)
0923 22:10:28.263289 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.263319 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":127,"threads_started":1}
I20260923 22:10:28.370395 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:28.370191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:38643)
0923 22:10:28.370310 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:28.370341 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260923 22:10:29.203550 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.203336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:48107)
0923 22:10:29.203469 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.203502 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260923 22:10:29.263935 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.263738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:51943)
0923 22:10:29.263867 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.263900 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260923 22:10:29.370949 16954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:29.370770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:38413)
0923 22:10:29.370883 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:29.370916 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260923 22:10:30.204133 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.203937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:50663)
0923 22:10:30.204059 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.204090 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":50,"threads_started":1}
I20260923 22:10:30.264485 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.264323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:56779)
0923 22:10:30.264420 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.264454 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":47,"threads_started":1}
I20260923 22:10:30.371637 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:30.371376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:51323)
0923 22:10:30.371552 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:30.371589 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
I20260923 22:10:30.749516 16958 raft_consensus.cc:482] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:30.749590 16958 raft_consensus.cc:504] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91f236bd7cde47b1910f7c995ecf3387" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 45651 } } peers { permanent_uuid: "c931c70f66584cbfb005c5b9b1d1ac26" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 35431 } } peers { permanent_uuid: "a6bbeeba9b7f4eca9308fa1733d1bab9" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37407 } }
I20260923 22:10:30.749723 16958 leader_election.cc:288] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651), c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431)
W20260923 22:10:30.750046 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c931c70f66584cbfb005c5b9b1d1ac26 (127.1.161.130:35431): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
W20260923 22:10:30.750105 16532 leader_election.cc:341] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 91f236bd7cde47b1910f7c995ecf3387 (127.1.161.131:45651): Not found: Tablet not found: 1ec0b10f9f0448a88c85964b7fa26d1f
I20260923 22:10:30.750120 16532 leader_election.cc:302] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6bbeeba9b7f4eca9308fa1733d1bab9; no voters: 91f236bd7cde47b1910f7c995ecf3387, c931c70f66584cbfb005c5b9b1d1ac26
I20260923 22:10:30.750180 16958 raft_consensus.cc:2726] T 1ec0b10f9f0448a88c85964b7fa26d1f P a6bbeeba9b7f4eca9308fa1733d1bab9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:10:31.204766 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:31.204559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:41337)
0923 22:10:31.204690 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:31.204723 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260923 22:10:31.265014 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:31.264851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:36119)
0923 22:10:31.264952 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:31.264982 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":46,"threads_started":1}
I20260923 22:10:31.372257 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:31.372067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:42773)
0923 22:10:31.372186 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:31.372218 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260923 22:10:32.205426 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.205247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:42977)
0923 22:10:32.205355 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.205385 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
I20260923 22:10:32.265560 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.265400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:42769)
0923 22:10:32.265506 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.265533 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260923 22:10:32.372849 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:32.372644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:57139)
0923 22:10:32.372766 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:32.372814 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260923 22:10:33.206086 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.205878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.129:57013)
0923 22:10:33.205990 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.206019 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260923 22:10:33.266179 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.265999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.130:55943)
0923 22:10:33.266119 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.266146 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260923 22:10:33.370829 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16486
I20260923 22:10:33.373485 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:33.373285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:44727 (local address 127.1.161.131:42493)
0923 22:10:33.373406 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:33.373440 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":50,"threads_started":1}
I20260923 22:10:33.375563 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16649
I20260923 22:10:33.379256 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16782
2026-09-23T22:10:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (17980 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:10:33 dist-test-slave-npvd krb5kdc[16970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:10:33 dist-test-slave-npvd krb5kdc[16970](info): set up 2 sockets
Sep 23 22:10:33 dist-test-slave-npvd krb5kdc[16970](info): commencing operation
krb5kdc: starting...
W20260923 22:10:34.310891 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:10:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:10:34.416240 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:34.507827 16983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:34.508054 16983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:34.508085 16983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:34.509076 16983 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:10:34.509135 16983 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:10:34.509155 16983 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:10:34.509172 16983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:34.509199 16983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:34.509229 16983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:34.509251 16983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:34.509279 16983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:34.510588 16983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:34.510838 16983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:34.511071 16983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:34.515616 16991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:34.515655 16988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:34.515714 16983 server_base.cc:1034] running on GCE node
W20260923 22:10:34.515622 16989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:34.516016 16983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:34.516273 16983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:34.517431 16983 hybrid_clock.cc:648] HybridClock initialized: now 1790201434517410 us; error 33 us; skew 500 ppm
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:34.519671 16983 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:34.520112 16983 webserver.cc:460] Webserver started at http://127.1.161.190:46827/ using document root <none> and password file <none>
I20260923 22:10:34.520314 16983 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:34.520375 16983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:34.520498 16983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:34.521443 16983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "69130788682a495a9b8db93c465f1225"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "20d4fb9114c78666074caebb76607c90"
server_key_iv: "d94f86f3c802db026276619ac43f2a25"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.521703 16983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "69130788682a495a9b8db93c465f1225"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "20d4fb9114c78666074caebb76607c90"
server_key_iv: "d94f86f3c802db026276619ac43f2a25"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.522936 16983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:10:34.523670 16998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.523823 16983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.523898 16983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "69130788682a495a9b8db93c465f1225"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "20d4fb9114c78666074caebb76607c90"
server_key_iv: "d94f86f3c802db026276619ac43f2a25"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.523962 16983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:34.560554 16983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:34.562173 16983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:34.562317 16983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:34.566118 16983 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:35087
I20260923 22:10:34.566184 17050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:35087 every 8 connection(s)
I20260923 22:10:34.566452 16983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:34.567235 17051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:34.569233 17051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap starting.
I20260923 22:10:34.569849 17051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:34.570119 17051 log.cc:824] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:34.570420 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 16983
I20260923 22:10:34.570515 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:10:34.570706 1670 external_mini_cluster.cc:1428] Setting key 0afed1bb3eedac4c2d6684915c4a56ba
I20260923 22:10:34.570828 17051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: No bootstrap required, opened a new log
I20260923 22:10:34.572476 17051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:34.572633 17051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:34.572670 17051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69130788682a495a9b8db93c465f1225, State: Initialized, Role: FOLLOWER
I20260923 22:10:34.572832 17051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:34.572923 17051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:34.572973 17051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:34.573040 17051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:34.573815 17051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:34.573887 17051 leader_election.cc:302] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69130788682a495a9b8db93c465f1225; no voters:
I20260923 22:10:34.574059 17051 leader_election.cc:288] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:10:34.574082 17056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:34.574172 17056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 LEADER]: Becoming Leader. State: Replica: 69130788682a495a9b8db93c465f1225, State: Running, Role: LEADER
I20260923 22:10:34.574281 17051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:34.574294 17056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:34.574689 17057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69130788682a495a9b8db93c465f1225. Latest consensus state: current_term: 1 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:10:34.574795 17057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:34.575235 17058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:10:34.575315 17058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:34.575649 17063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:34.576332 17063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:34.578500 17063 catalog_manager.cc:1347] Generated new cluster ID: 04f8f7b986654dc78147e23c8fb4798c
I20260923 22:10:34.578545 17063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:34.582393 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:34.571142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.1.161.190:35087)
0923 22:10:34.571311 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:34.571318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:34.571348 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:10:34.572460 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:34.572466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:34.572511 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:34.572617 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:34.573382 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.574319 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.575449 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.576072 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.576141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:34.579135 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:34.579139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:34.579139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:34.579149 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:34.580873 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.581194 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.581196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.581197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.581307 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.581481 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.581482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.581482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.581734 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:34.581939 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:34.582030 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:10:34.582202 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260923 22:10:34.595360 17063 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:10:34.595816 17063 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:34.603274 17063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Generated new TSK 0
I20260923 22:10:34.603426 17063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:10:34.629392 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:34.718436 17077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:34.718632 17077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:34.718653 17077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:34.719637 17077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:34.719668 17077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:34.719692 17077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:34.721020 17077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:34.721179 17077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:34.721320 17077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:34.725837 17083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:34.725867 17077 server_base.cc:1034] running on GCE node
W20260923 22:10:34.725862 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:34.725828 17082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:34.726287 17077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:34.726524 17077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:34.727661 17077 hybrid_clock.cc:648] HybridClock initialized: now 1790201434727653 us; error 24 us; skew 500 ppm
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:34.729779 17077 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:34.730144 17077 webserver.cc:460] Webserver started at http://127.1.161.129:46663/ using document root <none> and password file <none>
I20260923 22:10:34.730290 17077 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:34.730322 17077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:34.730394 17077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:34.731226 17077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "d60ab4b2740844ffba07e609a38ed085"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7b04d903da893b47521160f963e0cd09"
server_key_iv: "0a69ee285f967a78f4730f2ebf64afe5"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.731482 17077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "d60ab4b2740844ffba07e609a38ed085"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7b04d903da893b47521160f963e0cd09"
server_key_iv: "0a69ee285f967a78f4730f2ebf64afe5"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.732657 17077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:34.733258 17092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.733398 17077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:34.733470 17077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d60ab4b2740844ffba07e609a38ed085"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7b04d903da893b47521160f963e0cd09"
server_key_iv: "0a69ee285f967a78f4730f2ebf64afe5"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.733515 17077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:34.744905 17077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:34.746299 17077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:34.746387 17077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:34.746526 17077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:34.746721 17077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:34.746747 17077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.746762 17077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:34.746773 17077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.751611 17077 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46725
I20260923 22:10:34.751698 17205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46725 every 8 connection(s)
I20260923 22:10:34.751976 17077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:34.753747 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17077
I20260923 22:10:34.753841 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:10:34.754024 1670 external_mini_cluster.cc:1428] Setting key 512ef329f0a3116d783b4ad349cae723
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:34.760579 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:34.752723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:49095 (local address 127.1.161.190:35087)
0923 22:10:34.752783 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:34.752785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:34.752913 (+ 128us) server_negotiation.cc:407] Connection header received
0923 22:10:34.753288 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:34.753289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:34.753299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:34.753342 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:34.754364 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.754928 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.755927 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.756129 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.756204 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:34.758179 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:34.758181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:34.758181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:34.758189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:34.759425 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.759779 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.759780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.759780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.759831 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.760042 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.760043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.760043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.760138 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:34.760199 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:34.760428 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 22:10:34.760503 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:10:34.760713 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:34.752638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:49095)
0923 22:10:34.752789 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:34.752826 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:34.753219 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:34.753387 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:34.753390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:34.754049 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:34.754268 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:34.754272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.755015 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.755016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:34.755824 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:34.755827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.756215 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.756216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:34.756324 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:34.756755 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:34.756763 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:34.758095 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:34.759508 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:34.759520 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:34.759521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:34.759703 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:34.759889 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:34.759890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:34.759890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:34.759981 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:34.760201 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:34.760207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:34.760317 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:10:34.760398 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 22:10:34.760528 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
I20260923 22:10:34.761102 17206 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:34.761189 17206 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:34.761343 17206 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:34.761785 17015 ts_manager.cc:194] Registered new tserver with Master: d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725)
I20260923 22:10:34.762491 17015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:49095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:10:34.777370 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:34.847491 17211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:34.847638 17211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:34.847657 17211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:34.848555 17211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:34.848585 17211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:34.848608 17211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:34.849725 17211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:34.849865 17211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:34.850003 17211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:10:34.854446 17211 server_base.cc:1034] running on GCE node
W20260923 22:10:34.854426 17217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:34.854418 17219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:34.854422 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:34.854827 17211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:34.855000 17211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:34.856124 17211 hybrid_clock.cc:648] HybridClock initialized: now 1790201434856085 us; error 54 us; skew 500 ppm
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:34.858201 17211 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:34.858600 17211 webserver.cc:460] Webserver started at http://127.1.161.130:46713/ using document root <none> and password file <none>
I20260923 22:10:34.858783 17211 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:34.858839 17211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:34.858959 17211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:34.859822 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "9287851cf1d048398c74fa4640a6a95f"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7771a4f3e36279a0a5ba8849c57e1536"
server_key_iv: "404e570014c7764e9f72f3f35be28374"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.860121 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "9287851cf1d048398c74fa4640a6a95f"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7771a4f3e36279a0a5ba8849c57e1536"
server_key_iv: "404e570014c7764e9f72f3f35be28374"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.861199 17211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:10:34.861799 17226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.861985 17211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:34.862082 17211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "9287851cf1d048398c74fa4640a6a95f"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7771a4f3e36279a0a5ba8849c57e1536"
server_key_iv: "404e570014c7764e9f72f3f35be28374"
server_key_version: "encryptionkey@0"
I20260923 22:10:34.862162 17211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:34.899201 17211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:34.901331 17211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:34.901484 17211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:34.901700 17211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:34.902021 17211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:34.902077 17211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.902110 17211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:34.902137 17211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:34.907513 17211 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:36321
I20260923 22:10:34.907580 17339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:36321 every 8 connection(s)
I20260923 22:10:34.907836 17211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:34.911311 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17211
I20260923 22:10:34.911417 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:10:34.911593 1670 external_mini_cluster.cc:1428] Setting key 5d5b8ed9c948538a8f90a263ef543f1c
Sep 23 22:10:34 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201434, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:34.916466 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:34.908407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:40263 (local address 127.1.161.190:35087)
0923 22:10:34.908478 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:34.908481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:34.908687 (+ 206us) server_negotiation.cc:407] Connection header received
0923 22:10:34.909149 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:34.909150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:34.909177 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:34.909233 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:34.909994 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.910412 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.911260 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.911485 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.911537 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:34.913774 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:34.913775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:34.913776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:34.913783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:34.915145 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.915557 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.915559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.915559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.915624 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:34.915857 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:34.915858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:34.915859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:34.915970 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:34.916046 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:34.916306 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 22:10:34.916381 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 22:10:34.916636 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:34.908381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:40263)
0923 22:10:34.908577 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:34.908611 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:34.909042 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:34.909285 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:34.909288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:34.909773 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:34.909929 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:34.909933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.910478 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.910491 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:34.911189 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:34.911191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:34.911560 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:34.911561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:34.911693 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:34.912231 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:34.912241 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:34.913695 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:34.915277 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:34.915282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:34.915284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:34.915481 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:34.915692 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:34.915692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:34.915693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:34.915784 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:34.916050 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:34.916056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:34.916171 (+ 115us) client_negotiation.cc:769] Sending connection context
0923 22:10:34.916285 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:10:34.916423 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260923 22:10:34.917141 17340 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:34.917259 17340 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:34.917459 17340 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:34.917789 17015 ts_manager.cc:194] Registered new tserver with Master: 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321)
I20260923 22:10:34.918133 17015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:40263
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:10:34.935055 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:35.005334 17345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:35.005497 17345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:35.005517 17345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:35.006412 17345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:35.006439 17345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:35.006460 17345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:35.007727 17345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:35.007881 17345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:35.008026 17345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:10:35.012254 17345 server_base.cc:1034] running on GCE node
W20260923 22:10:35.012254 17353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:35.012238 17351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:35.012238 17350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:35.012588 17345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:35.012811 17345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:35.013971 17345 hybrid_clock.cc:648] HybridClock initialized: now 1790201435013967 us; error 39 us; skew 500 ppm
Sep 23 22:10:35 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201435, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:35.016039 17345 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:35.016433 17345 webserver.cc:460] Webserver started at http://127.1.161.131:35785/ using document root <none> and password file <none>
I20260923 22:10:35.016620 17345 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:35.016678 17345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:35.016804 17345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:10:35.017663 17345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "7519c85f99014678b012510539c95ba6"
format_stamp: "Formatted at 2026-09-23 22:10:35 on dist-test-slave-npvd"
server_key: "e3c314ab85345d1050919ea6456de8ad"
server_key_iv: "07dc9ae86664b17dd1611d95c33a619b"
server_key_version: "encryptionkey@0"
I20260923 22:10:35.018013 17345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "7519c85f99014678b012510539c95ba6"
format_stamp: "Formatted at 2026-09-23 22:10:35 on dist-test-slave-npvd"
server_key: "e3c314ab85345d1050919ea6456de8ad"
server_key_iv: "07dc9ae86664b17dd1611d95c33a619b"
server_key_version: "encryptionkey@0"
I20260923 22:10:35.019301 17345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:10:35.019950 17360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:35.020098 17345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:35.020184 17345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "7519c85f99014678b012510539c95ba6"
format_stamp: "Formatted at 2026-09-23 22:10:35 on dist-test-slave-npvd"
server_key: "e3c314ab85345d1050919ea6456de8ad"
server_key_iv: "07dc9ae86664b17dd1611d95c33a619b"
server_key_version: "encryptionkey@0"
I20260923 22:10:35.020262 17345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:35.047952 17345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:35.049540 17345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:35.049675 17345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:35.049860 17345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:35.050114 17345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:35.050163 17345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:35.050191 17345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:35.050213 17345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:35.055547 17345 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:41103
I20260923 22:10:35.055578 17473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:41103 every 8 connection(s)
I20260923 22:10:35.055845 17345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:35.058923 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17345
I20260923 22:10:35.059024 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:10:35.059219 1670 external_mini_cluster.cc:1428] Setting key c9e93e81af1e773a7abbb48c6f47c287
Sep 23 22:10:35 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201435, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:35.063874 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.056477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:58995 (local address 127.1.161.190:35087)
0923 22:10:35.056548 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.056550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.056717 (+ 167us) server_negotiation.cc:407] Connection header received
0923 22:10:35.057095 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.057097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.057106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.057153 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:35.057981 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.058408 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.059456 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.059739 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.059802 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.061656 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:35.061657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:35.061657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:35.061662 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:35.062804 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:35.063197 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:35.063198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:35.063199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:35.063251 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:35.063444 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:35.063445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:35.063445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:35.063529 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:35.063573 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.063764 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.063822 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:10:35.064005 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.056462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:58995)
0923 22:10:35.056615 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.056643 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.057018 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.057228 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.057231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.057748 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:35.057908 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.057912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.058477 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.058478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.059331 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.059334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.059818 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.059819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.059945 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.060433 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:35.060440 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:35.061588 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:35.062872 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:35.062878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:35.062879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:35.063118 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:35.063309 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:35.063310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:35.063310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:35.063394 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:35.063580 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:35.063583 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:35.063670 (+ 87us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.063738 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.063832 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260923 22:10:35.064406 17474 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:35.064481 17474 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:35.064631 17474 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:35.064952 17015 ts_manager.cc:194] Registered new tserver with Master: 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
I20260923 22:10:35.065231 17015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:58995
I20260923 22:10:35.071301 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:35.078106 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.072703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.1.161.190:35087)
0923 22:10:35.072770 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.072772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.072880 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:10:35.072954 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.072954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.072964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.073034 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:35.073536 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.073985 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.074727 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.074937 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.075012 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.076070 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:35.076071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:35.076071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:35.076077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:35.077026 (+ 949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:35.077361 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:35.077362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:35.077362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:35.077411 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:35.077612 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:35.077613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:35.077613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:35.077740 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:35.077841 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.077947 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.078019 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:10:35.079334 17015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:35.080634 17015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:35.085930 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36321 (local address 127.0.0.1:43868)
0923 22:10:35.082669 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.082708 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.082770 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.083011 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.083013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.083016 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:35.083099 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.083100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084275 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.084276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.085005 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.085007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.085681 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.085682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.085755 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.085768 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.085831 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.085871 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":43,"thread_start_us":61,"threads_started":1}
I20260923 22:10:35.085961 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.1.161.130:36321)
0923 22:10:35.082857 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.082861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.082875 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:10:35.082906 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.082907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.082919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.083006 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:35.083203 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084160 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.085089 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.085609 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.085675 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.085718 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.085854 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.085892 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260923 22:10:35.086328 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41103 (local address 127.0.0.1:45622)
0923 22:10:35.083212 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.083221 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.083263 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.083471 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.083472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.083473 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:35.083506 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.083508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084196 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.084197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.085310 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.085312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.086109 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.086110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.086158 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.086161 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.086230 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.086268 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":73,"thread_start_us":46,"threads_started":1}
I20260923 22:10:35.086499 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.1.161.131:41103)
0923 22:10:35.083045 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.083048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.083311 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:10:35.083344 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.083346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.083357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.083416 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:35.083556 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084102 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.085384 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.085972 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.086033 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.086080 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.086245 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.086276 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":68,"threads_started":1}
I20260923 22:10:35.086621 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.0.0.1:57490)
0923 22:10:35.082866 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.082877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.082933 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.083130 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.083131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.083132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:35.083193 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.083194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084237 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.084238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.085504 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.085506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.086428 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.086429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.086484 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.086487 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.086531 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.086566 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":73,"thread_start_us":48,"threads_started":1}
I20260923 22:10:35.086838 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.082758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.1.161.129:46725)
0923 22:10:35.082925 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.082930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.082941 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:10:35.083036 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.083038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.083056 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.083152 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:35.083242 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.084158 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.085585 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.086292 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.086349 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.086398 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.086723 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.086767 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260923 22:10:35.086840 17274 tablet_service.cc:1467] Processing CreateTablet for tablet d5166b913a58403c9f6135812b6f0f71 (DEFAULT_TABLE table=test-table [id=2e6d03f20c78460d9e620f8ff52f2465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:35.087165 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5166b913a58403c9f6135812b6f0f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:35.087535 17140 tablet_service.cc:1467] Processing CreateTablet for tablet d5166b913a58403c9f6135812b6f0f71 (DEFAULT_TABLE table=test-table [id=2e6d03f20c78460d9e620f8ff52f2465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:35.087698 17408 tablet_service.cc:1467] Processing CreateTablet for tablet d5166b913a58403c9f6135812b6f0f71 (DEFAULT_TABLE table=test-table [id=2e6d03f20c78460d9e620f8ff52f2465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:35.087832 17140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5166b913a58403c9f6135812b6f0f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:35.087915 17408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5166b913a58403c9f6135812b6f0f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:35.089834 17488 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Bootstrap starting.
I20260923 22:10:35.090226 17489 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Bootstrap starting.
I20260923 22:10:35.090474 17488 tablet_bootstrap.cc:654] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:35.090799 17490 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Bootstrap starting.
I20260923 22:10:35.090975 17489 tablet_bootstrap.cc:654] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:35.090962 17488 log.cc:824] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:35.091405 17490 tablet_bootstrap.cc:654] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:35.091524 17489 log.cc:824] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:35.091719 17488 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: No bootstrap required, opened a new log
I20260923 22:10:35.091790 17488 ts_tablet_manager.cc:1397] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:35.091992 17490 log.cc:824] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:35.092300 17489 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: No bootstrap required, opened a new log
I20260923 22:10:35.092373 17489 ts_tablet_manager.cc:1397] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:10:35.092656 17490 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: No bootstrap required, opened a new log
I20260923 22:10:35.092736 17490 ts_tablet_manager.cc:1397] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:35.093689 17488 raft_consensus.cc:348] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.093849 17488 raft_consensus.cc:374] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:35.093880 17488 raft_consensus.cc:729] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7519c85f99014678b012510539c95ba6, State: Initialized, Role: FOLLOWER
I20260923 22:10:35.094030 17488 consensus_queue.cc:260] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.094278 17488 ts_tablet_manager.cc:1428] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:10:35.094260 17489 raft_consensus.cc:348] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.094386 17474 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
I20260923 22:10:35.094414 17489 raft_consensus.cc:374] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:35.094455 17489 raft_consensus.cc:729] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9287851cf1d048398c74fa4640a6a95f, State: Initialized, Role: FOLLOWER
I20260923 22:10:35.094576 17489 consensus_queue.cc:260] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.094842 17489 ts_tablet_manager.cc:1428] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:10:35.094887 17340 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
I20260923 22:10:35.094667 17490 raft_consensus.cc:348] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.095206 17490 raft_consensus.cc:374] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:35.095259 17490 raft_consensus.cc:729] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ab4b2740844ffba07e609a38ed085, State: Initialized, Role: FOLLOWER
I20260923 22:10:35.095496 17490 consensus_queue.cc:260] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.095801 17490 ts_tablet_manager.cc:1428] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 22:10:35.095826 17206 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
W20260923 22:10:35.158324 17341 tablet.cc:2369] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:10:35.252691 17207 tablet.cc:2369] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:35.292409 17495 raft_consensus.cc:482] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:35.292537 17495 raft_consensus.cc:504] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.292838 17495 leader_election.cc:288] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
I20260923 22:10:35.296243 17487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.293089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:40667 (local address 127.1.161.129:46725)
0923 22:10:35.293139 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.293141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.293148 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:35.293175 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.293176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.293183 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.293234 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:35.293412 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.294078 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295344 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.295845 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295919 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.295928 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.296145 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.296184 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:10:35.296243 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.293077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:54647 (local address 127.1.161.131:41103)
0923 22:10:35.293113 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.293115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.293283 (+ 168us) server_negotiation.cc:407] Connection header received
0923 22:10:35.293320 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.293321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.293331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.293368 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:35.293511 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.294092 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295344 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.295874 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295931 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.295939 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.296141 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.296180 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:10:35.296254 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.292996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.1.161.130:40667)
0923 22:10:35.293045 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.293068 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.293115 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.293260 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.293261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.293262 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:35.293328 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.293331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.294158 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.294159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.295267 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.295270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.295965 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.296063 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.296066 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.296117 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.296166 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":48}
I20260923 22:10:35.296260 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.292997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41103 (local address 127.1.161.130:54647)
0923 22:10:35.293209 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.293219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:35.293268 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:35.293386 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:35.293387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:35.293388 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:35.293438 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.293439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.294158 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.294159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.295267 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:35.295270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.295965 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.295966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:35.296048 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.296053 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:10:35.296117 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:10:35.296171 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":100,"thread_start_us":97,"threads_started":1}
I20260923 22:10:35.296778 17428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5166b913a58403c9f6135812b6f0f71" candidate_uuid: "9287851cf1d048398c74fa4640a6a95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7519c85f99014678b012510539c95ba6" is_pre_election: true
I20260923 22:10:35.296838 17160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5166b913a58403c9f6135812b6f0f71" candidate_uuid: "9287851cf1d048398c74fa4640a6a95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ab4b2740844ffba07e609a38ed085" is_pre_election: true
I20260923 22:10:35.296937 17428 raft_consensus.cc:2454] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9287851cf1d048398c74fa4640a6a95f in term 0.
I20260923 22:10:35.297036 17160 raft_consensus.cc:2454] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9287851cf1d048398c74fa4640a6a95f in term 0.
I20260923 22:10:35.297122 17230 leader_election.cc:302] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f; no voters:
I20260923 22:10:35.297286 17495 raft_consensus.cc:2781] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:35.297335 17495 raft_consensus.cc:482] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:35.297349 17495 raft_consensus.cc:3037] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:35.298048 17495 raft_consensus.cc:504] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.298168 17495 leader_election.cc:288] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [CANDIDATE]: Term 1 election: Requested vote from peers d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
I20260923 22:10:35.298317 17160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5166b913a58403c9f6135812b6f0f71" candidate_uuid: "9287851cf1d048398c74fa4640a6a95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ab4b2740844ffba07e609a38ed085"
I20260923 22:10:35.298341 17428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5166b913a58403c9f6135812b6f0f71" candidate_uuid: "9287851cf1d048398c74fa4640a6a95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7519c85f99014678b012510539c95ba6"
I20260923 22:10:35.298390 17160 raft_consensus.cc:3037] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:35.298399 17428 raft_consensus.cc:3037] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:35.299192 17428 raft_consensus.cc:2454] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9287851cf1d048398c74fa4640a6a95f in term 1.
I20260923 22:10:35.299190 17160 raft_consensus.cc:2454] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9287851cf1d048398c74fa4640a6a95f in term 1.
I20260923 22:10:35.299335 17230 leader_election.cc:302] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f; no voters:
I20260923 22:10:35.299526 17495 raft_consensus.cc:2781] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:35.299688 17495 raft_consensus.cc:686] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 1 LEADER]: Becoming Leader. State: Replica: 9287851cf1d048398c74fa4640a6a95f, State: Running, Role: LEADER
I20260923 22:10:35.299787 17495 consensus_queue.cc:237] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:35.300505 17014 catalog_manager.cc:5495] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9287851cf1d048398c74fa4640a6a95f (127.1.161.130). New cstate: current_term: 1 leader_uuid: "9287851cf1d048398c74fa4640a6a95f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } health_report { overall_health: UNKNOWN } } }
W20260923 22:10:35.306293 17475 tablet.cc:2369] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:35.313064 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.310833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.1.161.130:36321)
0923 22:10:35.310881 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:35.310883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:35.310890 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:10:35.310954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:35.310955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:35.310969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:35.311043 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:35.311196 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.311572 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.312302 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:35.312509 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:35.312592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:35.312754 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:35.312845 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:35.312935 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:35.312959 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 22:10:35.312977 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:10:35.314402 17160 raft_consensus.cc:1261] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Refusing update from remote peer 9287851cf1d048398c74fa4640a6a95f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:35.314409 17428 raft_consensus.cc:1261] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Refusing update from remote peer 9287851cf1d048398c74fa4640a6a95f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:35.314673 17498 consensus_queue.cc:1035] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:35.314790 17495 consensus_queue.cc:1035] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:35.316618 17501 mvcc.cc:205] Tried to move back new op lower bound from 7332665079045795840 to 7332665078988427264. Current Snapshot: MvccSnapshot[applied={T|T < 7332665079045795840}]
I20260923 22:10:35.316879 17502 mvcc.cc:205] Tried to move back new op lower bound from 7332665079045795840 to 7332665078988427264. Current Snapshot: MvccSnapshot[applied={T|T < 7332665079045795840}]
I20260923 22:10:35.318794 17014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:35.318871 17014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:35.319521 17014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Sending DeleteTablet for 3 replicas of tablet d5166b913a58403c9f6135812b6f0f71
I20260923 22:10:35.319831 17140 tablet_service.cc:1514] Processing DeleteTablet for tablet d5166b913a58403c9f6135812b6f0f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:35 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:57490
I20260923 22:10:35.319862 17408 tablet_service.cc:1514] Processing DeleteTablet for tablet d5166b913a58403c9f6135812b6f0f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:35 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:45622
I20260923 22:10:35.320019 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet d5166b913a58403c9f6135812b6f0f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:35 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:43868
I20260923 22:10:35.320050 17512 tablet_replica.cc:330] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: stopping tablet replica
I20260923 22:10:35.320138 17512 raft_consensus.cc:2229] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:35.320206 17513 tablet_replica.cc:330] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: stopping tablet replica
I20260923 22:10:35.320206 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17077
I20260923 22:10:35.320291 17512 raft_consensus.cc:2258] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:35.320312 17513 raft_consensus.cc:2229] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:35.320436 17513 raft_consensus.cc:2258] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:35.320735 17512 ts_tablet_manager.cc:1905] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:35.320847 17513 ts_tablet_manager.cc:1905] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:35.322290 17512 ts_tablet_manager.cc:1918] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:35.322417 17512 log.cc:1196] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/d5166b913a58403c9f6135812b6f0f71
I20260923 22:10:35.322746 17512 ts_tablet_manager.cc:1939] T d5166b913a58403c9f6135812b6f0f71 P 7519c85f99014678b012510539c95ba6: Deleting consensus metadata
I20260923 22:10:35.322772 17513 ts_tablet_manager.cc:1918] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:35.322873 17513 log.cc:1196] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/d5166b913a58403c9f6135812b6f0f71
I20260923 22:10:35.323233 17513 ts_tablet_manager.cc:1939] T d5166b913a58403c9f6135812b6f0f71 P 9287851cf1d048398c74fa4640a6a95f: Deleting consensus metadata
I20260923 22:10:35.323589 17000 catalog_manager.cc:4841] TS 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): tablet d5166b913a58403c9f6135812b6f0f71 (table test-table [id=2e6d03f20c78460d9e620f8ff52f2465]) successfully deleted
I20260923 22:10:35.323907 17002 catalog_manager.cc:4841] TS 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): tablet d5166b913a58403c9f6135812b6f0f71 (table test-table [id=2e6d03f20c78460d9e620f8ff52f2465]) successfully deleted
W20260923 22:10:35.326890 16999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:46725 (error 108)
I20260923 22:10:35.327066 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17211
I20260923 22:10:35.327668 17482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:35.327260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.0.0.1:57498)
0923 22:10:35.327595 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:35.327635 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318}
W20260923 22:10:35.328783 16999 catalog_manager.cc:4584] TS d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5166b913a58403c9f6135812b6f0f71: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46725: connect: Connection refused (error 111)
I20260923 22:10:35.331120 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17345
I20260923 22:10:35.335810 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 16983
I20260923 22:10:55.340909 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--webserver_interface=127.1.161.190
--webserver_port=46827
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:10:55.411829 17515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:55.411999 17515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:55.412019 17515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:55.413059 17515 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:10:55.413095 17515 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:10:55.413107 17515 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:10:55.413149 17515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:10:55.413167 17515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:55.413192 17515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:55.413214 17515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:10:55.413240 17515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:10:55.414638 17515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:55.414872 17515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:55.415098 17515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:55.422861 17524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.422861 17521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.422881 17522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.422982 17515 server_base.cc:1034] running on GCE node
I20260923 22:10:55.423363 17515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:55.423616 17515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:55.424767 17515 hybrid_clock.cc:648] HybridClock initialized: now 1790201455424742 us; error 36 us; skew 500 ppm
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:55.426887 17515 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:10:55.427314 17515 webserver.cc:460] Webserver started at http://127.1.161.190:46827/ using document root <none> and password file <none>
I20260923 22:10:55.427511 17515 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:55.427579 17515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:55.428711 17515 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:10:55.429239 17531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.429380 17515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.429466 17515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "69130788682a495a9b8db93c465f1225"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "20d4fb9114c78666074caebb76607c90"
server_key_iv: "d94f86f3c802db026276619ac43f2a25"
server_key_version: "encryptionkey@0"
I20260923 22:10:55.429776 17515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:55.449783 17515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:55.451269 17515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:55.451398 17515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:55.454769 17515 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:35087
I20260923 22:10:55.454814 17583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:35087 every 8 connection(s)
I20260923 22:10:55.455281 17515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:10:55.456436 17584 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:10:55.457016 17584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap starting.
I20260923 22:10:55.464701 17584 log.cc:824] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:55.465250 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17515
I20260923 22:10:55.465515 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:46725
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=46663
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:55.466777 17584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:55.467041 17584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap complete.
I20260923 22:10:55.469017 17584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:55.469305 17584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69130788682a495a9b8db93c465f1225, State: Initialized, Role: FOLLOWER
I20260923 22:10:55.469445 17584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:55.469494 17584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:10:55.469547 17584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:10:55.469588 17584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 1 FOLLOWER]: Advancing to term 2
I20260923 22:10:55.470420 17584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:55.470491 17584 leader_election.cc:302] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69130788682a495a9b8db93c465f1225; no voters:
I20260923 22:10:55.470652 17584 leader_election.cc:288] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 22:10:55.470683 17588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 22:10:55.470780 17588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 LEADER]: Becoming Leader. State: Replica: 69130788682a495a9b8db93c465f1225, State: Running, Role: LEADER
I20260923 22:10:55.470873 17584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:10:55.470897 17588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:10:55.471138 17588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69130788682a495a9b8db93c465f1225. Latest consensus state: current_term: 2 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:10:55.471262 17588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:55.471141 17589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:10:55.471436 17589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:10:55.471508 17596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:10:55.472234 17596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:10:55.472357 17596 catalog_manager.cc:1259] Loaded cluster ID: 04f8f7b986654dc78147e23c8fb4798c
I20260923 22:10:55.472448 17596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:10:55.473603 17596 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:10:55.473819 17596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Loaded TSK: 0
I20260923 22:10:55.489091 17596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Generated new TSK 1
I20260923 22:10:55.489152 17596 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:10:55.537492 17587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:55.537675 17587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:55.537698 17587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:55.538745 17587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:55.538780 17587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:55.538802 17587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:10:55.540213 17587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:46725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=46663
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:55.540428 17587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:55.540635 17587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:55.544984 17612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.544989 17611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.545068 17614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.545329 17587 server_base.cc:1034] running on GCE node
I20260923 22:10:55.545526 17587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:55.545764 17587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:55.546909 17587 hybrid_clock.cc:648] HybridClock initialized: now 1790201455546891 us; error 35 us; skew 500 ppm
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:55.549360 17587 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:10:55.549846 17587 webserver.cc:460] Webserver started at http://127.1.161.129:46663/ using document root <none> and password file <none>
I20260923 22:10:55.550055 17587 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:55.550123 17587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:55.551621 17587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:55.552331 17621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.552493 17587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:10:55.552563 17587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d60ab4b2740844ffba07e609a38ed085"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7b04d903da893b47521160f963e0cd09"
server_key_iv: "0a69ee285f967a78f4730f2ebf64afe5"
server_key_version: "encryptionkey@0"
I20260923 22:10:55.552842 17587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:55.564143 17587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:55.565982 17587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:55.566097 17587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:55.566339 17587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:55.566704 17628 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:10:55.568393 17587 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:10:55.568445 17587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:55.568483 17587 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:10:55.568998 17587 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:10:55.569048 17587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:55.569110 17628 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Bootstrap starting.
I20260923 22:10:55.575695 17587 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46725
I20260923 22:10:55.575999 17587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:10:55.579284 17628 log.cc:824] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:55.579537 17735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46725 every 8 connection(s)
I20260923 22:10:55.579877 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17587
I20260923 22:10:55.580152 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:36321
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=46713
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:10:55.581779 17628 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:10:55.582046 17628 tablet_bootstrap.cc:492] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Bootstrap complete.
I20260923 22:10:55.582942 17628 ts_tablet_manager.cc:1397] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260923 22:10:55.584517 17628 raft_consensus.cc:348] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:55.584753 17628 raft_consensus.cc:729] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ab4b2740844ffba07e609a38ed085, State: Initialized, Role: FOLLOWER
I20260923 22:10:55.584924 17628 consensus_queue.cc:260] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } }
I20260923 22:10:55.585150 17628 ts_tablet_manager.cc:1428] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:55.592365 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.576660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:50939)
0923 22:10:55.579763 (+ 3103us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.579797 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.580301 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.580946 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.580950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.581264 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:55.581517 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.581522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.583073 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.583073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.584252 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.584254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.587255 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.587256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.587370 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.587886 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:55.587893 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:55.589137 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:55.590945 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.590952 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.590954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.591157 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.591422 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.591423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.591423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.591528 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.591841 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:55.591855 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:55.591969 (+ 114us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.592054 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.592199 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3069,"thread_start_us":104,"threads_started":1}
I20260923 22:10:55.592394 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.576715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:50939 (local address 127.1.161.190:35087)
0923 22:10:55.580496 (+ 3781us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.580504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.580536 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:10:55.580649 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.580655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.580702 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.580820 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:55.581639 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.582957 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.584346 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.584953 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.585035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.589264 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:55.589268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:55.589268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:55.589286 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:55.590857 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.591266 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.591269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.591270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.591345 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.591591 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.591592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.591592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.591771 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:55.591846 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.592110 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.592213 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3738,"thread_start_us":77,"threads_started":1}
I20260923 22:10:55.593148 17736 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:55.593247 17736 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:55.593403 17736 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:55.594064 17548 ts_manager.cc:194] Registered new tserver with Master: d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725)
I20260923 22:10:55.595118 17548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:50939
I20260923 22:10:55.595912 17736 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
I20260923 22:10:55.601892 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.594807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.1.161.129:46725)
0923 22:10:55.599434 (+ 4627us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.599437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.599443 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:10:55.599467 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.599468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.599478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.599555 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:55.599664 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.600228 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.601106 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.601481 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.601553 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.601598 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.601768 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.601812 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4605,"thread_start_us":73,"threads_started":1}
I20260923 22:10:55.601903 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.594600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.0.0.1:48988)
0923 22:10:55.599265 (+ 4665us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.599291 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.599339 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.599559 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.599562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.599563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:55.599614 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.599615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.600306 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.600307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.601022 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.601024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.601563 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.601564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.601634 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.601645 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.601751 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.601817 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4637,"thread_start_us":77,"threads_started":1}
I20260923 22:10:55.602501 17667 tablet_service.cc:1514] Processing DeleteTablet for tablet d5166b913a58403c9f6135812b6f0f71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:35 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:48988
I20260923 22:10:55.602957 17745 tablet_replica.cc:330] stopping tablet replica
I20260923 22:10:55.603070 17745 raft_consensus.cc:2229] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:55.603140 17745 raft_consensus.cc:2258] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:55.603404 17745 ts_tablet_manager.cc:1905] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:55.605289 17745 ts_tablet_manager.cc:1918] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:55.605365 17745 log.cc:1196] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal/wals/d5166b913a58403c9f6135812b6f0f71
I20260923 22:10:55.605621 17745 ts_tablet_manager.cc:1939] T d5166b913a58403c9f6135812b6f0f71 P d60ab4b2740844ffba07e609a38ed085: Deleting consensus metadata
I20260923 22:10:55.606113 17532 catalog_manager.cc:4841] TS d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725): tablet d5166b913a58403c9f6135812b6f0f71 (table test-table [id=2e6d03f20c78460d9e620f8ff52f2465]) successfully deleted
W20260923 22:10:55.661808 17741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:55.661969 17741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:55.661989 17741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:55.662952 17741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:55.662987 17741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:55.663012 17741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:10:55.664458 17741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:36321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=46713
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:55.664631 17741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:55.664786 17741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:55.669405 17750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.669456 17751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.669482 17741 server_base.cc:1034] running on GCE node
W20260923 22:10:55.669593 17753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.669885 17741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:55.670063 17741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:55.671216 17741 hybrid_clock.cc:648] HybridClock initialized: now 1790201455671198 us; error 32 us; skew 500 ppm
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:55.673703 17741 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:10:55.674110 17741 webserver.cc:460] Webserver started at http://127.1.161.130:46713/ using document root <none> and password file <none>
I20260923 22:10:55.674324 17741 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:55.674405 17741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:55.675688 17741 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:10:55.676244 17760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.676447 17741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:10:55.676535 17741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "9287851cf1d048398c74fa4640a6a95f"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7771a4f3e36279a0a5ba8849c57e1536"
server_key_iv: "404e570014c7764e9f72f3f35be28374"
server_key_version: "encryptionkey@0"
I20260923 22:10:55.676806 17741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:55.688441 17741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:55.689880 17741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:55.690012 17741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:55.690192 17741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:55.690438 17741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:55.690485 17741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.690517 17741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:55.690541 17741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.695631 17741 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:36321
I20260923 22:10:55.695716 17873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:36321 every 8 connection(s)
I20260923 22:10:55.695968 17741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:10:55.697642 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17741
I20260923 22:10:55.697923 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:41103
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=35785
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:55.705268 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.696530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:51769)
0923 22:10:55.696684 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.696729 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.697166 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.697405 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.697408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.698032 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:55.698235 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.698239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.698911 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.698912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.700157 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.700159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.700610 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.700611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.700706 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.701153 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:55.701159 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:55.702366 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:55.704069 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.704074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.704076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.704275 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.704495 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.704496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.704496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.704582 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.704866 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:55.704870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:55.704978 (+ 108us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.705047 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.705142 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260923 22:10:55.705304 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.696536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:51769 (local address 127.1.161.190:35087)
0923 22:10:55.696577 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.696579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.696791 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:10:55.697280 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.697281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.697297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.697345 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:55.698325 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.698818 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.700237 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.700534 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.700597 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.702450 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:55.702451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:55.702452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:55.702459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:55.703990 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.704352 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.704353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.704354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.704426 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.704646 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.704647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.704647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.704796 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:55.704872 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.705093 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.705183 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:10:55.705739 17874 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:55.705840 17874 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:55.706014 17874 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:55.706338 17548 ts_manager.cc:194] Registered new tserver with Master: 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321)
I20260923 22:10:55.706670 17548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:51769
W20260923 22:10:55.770606 17877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:10:55.770810 17877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:10:55.770830 17877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:10:55.772078 17877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:10:55.772113 17877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:10:55.772136 17877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:10:55.773460 17877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:41103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=35785
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:10:55.773650 17877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:10:55.773880 17877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:10:55.778469 17885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:10:55.778532 17883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.778579 17877 server_base.cc:1034] running on GCE node
W20260923 22:10:55.778518 17882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:10:55.778807 17877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:10:55.779067 17877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:10:55.780194 17877 hybrid_clock.cc:648] HybridClock initialized: now 1790201455780180 us; error 28 us; skew 500 ppm
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:10:55.782239 17877 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:10:55.782536 17877 webserver.cc:460] Webserver started at http://127.1.161.131:35785/ using document root <none> and password file <none>
I20260923 22:10:55.782691 17877 fs_manager.cc:359] Metadata directory not provided
I20260923 22:10:55.782733 17877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:10:55.783874 17877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260923 22:10:55.784368 17892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.784543 17877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.784619 17877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "7519c85f99014678b012510539c95ba6"
format_stamp: "Formatted at 2026-09-23 22:10:35 on dist-test-slave-npvd"
server_key: "e3c314ab85345d1050919ea6456de8ad"
server_key_iv: "07dc9ae86664b17dd1611d95c33a619b"
server_key_version: "encryptionkey@0"
I20260923 22:10:55.784871 17877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:10:55.790668 17877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:10:55.792565 17877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:10:55.792652 17877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:10:55.792831 17877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:10:55.793058 17877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:10:55.793085 17877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.793102 17877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:10:55.793114 17877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.798192 17877 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:41103
I20260923 22:10:55.798274 18005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:41103 every 8 connection(s)
I20260923 22:10:55.798525 17877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:10:55.802999 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 17877
Sep 23 22:10:55 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201455, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:10:55.806581 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.799130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:51869 (local address 127.1.161.190:35087)
0923 22:10:55.799215 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.799217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.799353 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:10:55.799743 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.799745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.799755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.799808 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:55.800530 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.800946 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.801765 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.801950 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.802017 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.804034 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:55.804036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:55.804036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:55.804044 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:55.805409 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.805751 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.805752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.805752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.805805 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.806040 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.806041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.806041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.806139 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:55.806199 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.806436 (+ 237us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.806506 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 22:10:55.806703 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.799105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:51869)
0923 22:10:55.799251 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.799290 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.799653 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.799860 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.799863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.800306 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:10:55.800449 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.800452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.801025 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.801025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.801672 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.801674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.802021 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.802022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.802123 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.802521 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:10:55.802528 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:10:55.803942 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:10:55.805496 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.805502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.805504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.805685 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.805877 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:10:55.805878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:10:55.805878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:10:55.805973 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:10:55.806202 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:10:55.806208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:10:55.806318 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.806405 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.806527 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260923 22:10:55.807099 18006 heartbeater.cc:347] Connected to a master server at 127.1.161.190:35087
I20260923 22:10:55.807209 18006 heartbeater.cc:464] Registering TS with master...
I20260923 22:10:55.807375 18006 heartbeater.cc:511] Master 127.1.161.190:35087 requested a full tablet report, sending...
I20260923 22:10:55.807715 17548 ts_manager.cc:194] Registered new tserver with Master: 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
I20260923 22:10:55.808082 17548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:51869
I20260923 22:10:55.809414 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.803356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.1.161.190:35087)
0923 22:10:55.803451 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.803452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.803565 (+ 113us) server_negotiation.cc:407] Connection header received
0923 22:10:55.803617 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.803618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.803630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.803691 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:10:55.804258 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.804725 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.805444 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.805620 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.805690 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.806972 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:10:55.806973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:10:55.806973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:10:55.806978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:10:55.808457 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.808752 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.808753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.808753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.808799 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:10:55.808991 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:10:55.808992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:10:55.808992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:10:55.809086 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:10:55.809235 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.809264 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.809328 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
I20260923 22:10:55.809993 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:10:55.812024 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.810257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.1.161.190:35087)
0923 22:10:55.810307 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.810308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.810378 (+ 70us) server_negotiation.cc:407] Connection header received
0923 22:10:55.810429 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.810430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.810450 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.810501 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:55.810560 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.810922 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.811544 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.811690 (+ 146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.811733 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.811842 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:55.811918 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:55.811965 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.811975 (+ 10us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.811989 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:10:55.812247 17548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:10:55.813616 17548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:10:55.815456 17667 tablet_service.cc:1467] Processing CreateTablet for tablet 17371d1ab7d740f7895291b07dad7fc2 (DEFAULT_TABLE table=test-table [id=329f796d616042c7b17d366cf156e6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:55.815627 17667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17371d1ab7d740f7895291b07dad7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:55.816932 17628 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Bootstrap starting.
I20260923 22:10:55.817265 17628 tablet_bootstrap.cc:654] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:55.818372 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.815202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36321 (local address 127.0.0.1:48652)
0923 22:10:55.815288 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.815297 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.815328 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.815552 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.815553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.815554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:55.815586 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.815587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.816379 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.816380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.817435 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.817436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.818123 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.818124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.818175 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.818178 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.818236 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.818317 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":34}
I20260923 22:10:55.818427 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.815290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48652 (local address 127.1.161.130:36321)
0923 22:10:55.815413 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.815417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.815426 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:10:55.815457 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.815459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.815473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.815557 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:55.815640 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.816294 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.817518 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.818029 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.818125 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.818175 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.818292 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.818337 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260923 22:10:55.818499 17628 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: No bootstrap required, opened a new log
I20260923 22:10:55.818533 17628 ts_tablet_manager.cc:1397] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260923 22:10:55.818619 17628 raft_consensus.cc:348] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.818686 17628 raft_consensus.cc:374] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:55.818722 17628 raft_consensus.cc:729] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ab4b2740844ffba07e609a38ed085, State: Initialized, Role: FOLLOWER
I20260923 22:10:55.818758 17628 consensus_queue.cc:260] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.818830 17628 ts_tablet_manager.cc:1428] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 22:10:55.819056 17808 tablet_service.cc:1467] Processing CreateTablet for tablet 17371d1ab7d740f7895291b07dad7fc2 (DEFAULT_TABLE table=test-table [id=329f796d616042c7b17d366cf156e6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:55.819362 17808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17371d1ab7d740f7895291b07dad7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:55.820102 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.815249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41103 (local address 127.0.0.1:38852)
0923 22:10:55.817046 (+ 1797us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.817055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.817100 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.817262 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.817263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.817264 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:55.817296 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.817297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.818163 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.818164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.819118 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.819120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.819909 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.819910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.819962 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.819965 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.820014 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.820047 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1780,"thread_start_us":74,"threads_started":1}
I20260923 22:10:55.820113 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.815390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.1.161.131:41103)
0923 22:10:55.815543 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.815547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.817101 (+ 1554us) server_negotiation.cc:407] Connection header received
0923 22:10:55.817163 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.817165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.817180 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.817255 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:55.817341 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.818070 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.819284 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.819833 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.819907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.819951 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.820025 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.820053 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260923 22:10:55.822094 18016 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Bootstrap starting.
I20260923 22:10:55.822713 18016 tablet_bootstrap.cc:654] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:55.822652 17940 tablet_service.cc:1467] Processing CreateTablet for tablet 17371d1ab7d740f7895291b07dad7fc2 (DEFAULT_TABLE table=test-table [id=329f796d616042c7b17d366cf156e6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:10:55.822926 17940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17371d1ab7d740f7895291b07dad7fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:10:55.823251 18016 log.cc:824] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:55.823931 18016 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: No bootstrap required, opened a new log
I20260923 22:10:55.824007 18016 ts_tablet_manager.cc:1397] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:10:55.825829 18016 raft_consensus.cc:348] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.825984 18016 raft_consensus.cc:374] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:55.826031 18016 raft_consensus.cc:729] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9287851cf1d048398c74fa4640a6a95f, State: Initialized, Role: FOLLOWER
I20260923 22:10:55.826049 18018 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Bootstrap starting.
I20260923 22:10:55.826150 18016 consensus_queue.cc:260] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.826416 18016 ts_tablet_manager.cc:1428] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:10:55.826431 17874 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
I20260923 22:10:55.826632 18018 tablet_bootstrap.cc:654] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Neither blocks nor log segments found. Creating new log.
I20260923 22:10:55.827118 18018 log.cc:824] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:10:55.827741 18018 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: No bootstrap required, opened a new log
I20260923 22:10:55.827818 18018 ts_tablet_manager.cc:1397] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:10:55.828936 18018 raft_consensus.cc:348] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.829043 18018 raft_consensus.cc:374] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:10:55.829061 18018 raft_consensus.cc:729] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7519c85f99014678b012510539c95ba6, State: Initialized, Role: FOLLOWER
I20260923 22:10:55.829203 18018 consensus_queue.cc:260] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.829463 18018 ts_tablet_manager.cc:1428] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:10:55.829471 18006 heartbeater.cc:503] Master 127.1.161.190:35087 was elected leader, sending a full tablet report...
W20260923 22:10:55.830036 17737 tablet.cc:2369] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:10:55.882530 18021 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:10:55.882639 18021 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.882911 18021 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725)
I20260923 22:10:55.885576 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.883051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.1.161.131:42337)
0923 22:10:55.883107 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.883118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.883154 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.883345 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.883346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.883348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:55.883420 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.883422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.884057 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.884058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.884794 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.884796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.885349 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.885350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.885420 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.885423 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.885468 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.885497 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":34}
I20260923 22:10:55.885596 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.883119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:42337 (local address 127.1.161.129:46725)
0923 22:10:55.883181 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.883183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.883218 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:10:55.883242 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.883242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.883253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.883303 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:55.883489 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.883982 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.884886 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.885278 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.885327 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.885333 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.885495 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.885538 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:10:55.885643 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.883132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:37963 (local address 127.1.161.130:36321)
0923 22:10:55.883181 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.883183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.883304 (+ 121us) server_negotiation.cc:407] Connection header received
0923 22:10:55.883371 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.883372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.883381 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.883435 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:10:55.883591 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.884034 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.884966 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.885302 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.885388 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.885394 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.885548 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.885582 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:10:55.885699 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.883067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36321 (local address 127.1.161.131:37963)
0923 22:10:55.883239 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.883249 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:10:55.883301 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:10:55.883447 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:10:55.883448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:10:55.883464 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:10:55.883531 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.883532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.884118 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.884119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.884823 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:10:55.884825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.885402 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.885403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:10:55.885475 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.885479 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:10:55.885531 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:10:55.885566 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260923 22:10:55.885959 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17371d1ab7d740f7895291b07dad7fc2" candidate_uuid: "7519c85f99014678b012510539c95ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ab4b2740844ffba07e609a38ed085" is_pre_election: true
I20260923 22:10:55.885989 17828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17371d1ab7d740f7895291b07dad7fc2" candidate_uuid: "7519c85f99014678b012510539c95ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9287851cf1d048398c74fa4640a6a95f" is_pre_election: true
I20260923 22:10:55.886093 17674 raft_consensus.cc:2454] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7519c85f99014678b012510539c95ba6 in term 0.
I20260923 22:10:55.886109 17828 raft_consensus.cc:2454] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7519c85f99014678b012510539c95ba6 in term 0.
I20260923 22:10:55.886303 17894 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f; no voters:
I20260923 22:10:55.886471 18021 raft_consensus.cc:2781] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:10:55.886509 18021 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:10:55.886523 18021 raft_consensus.cc:3037] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:55.887045 18021 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.887176 18021 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [CANDIDATE]: Term 1 election: Requested vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725)
I20260923 22:10:55.887319 17828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17371d1ab7d740f7895291b07dad7fc2" candidate_uuid: "7519c85f99014678b012510539c95ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9287851cf1d048398c74fa4640a6a95f"
I20260923 22:10:55.887319 17674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17371d1ab7d740f7895291b07dad7fc2" candidate_uuid: "7519c85f99014678b012510539c95ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60ab4b2740844ffba07e609a38ed085"
I20260923 22:10:55.887382 17828 raft_consensus.cc:3037] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:55.887391 17674 raft_consensus.cc:3037] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:10:55.888098 17674 raft_consensus.cc:2454] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7519c85f99014678b012510539c95ba6 in term 1.
I20260923 22:10:55.888098 17828 raft_consensus.cc:2454] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7519c85f99014678b012510539c95ba6 in term 1.
I20260923 22:10:55.888248 17893 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7519c85f99014678b012510539c95ba6, d60ab4b2740844ffba07e609a38ed085; no voters:
I20260923 22:10:55.888321 18021 raft_consensus.cc:2781] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:10:55.888453 18021 raft_consensus.cc:686] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 7519c85f99014678b012510539c95ba6, State: Running, Role: LEADER
I20260923 22:10:55.888536 18021 consensus_queue.cc:237] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:10:55.889207 17548 catalog_manager.cc:5495] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7519c85f99014678b012510539c95ba6 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "7519c85f99014678b012510539c95ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } health_report { overall_health: UNKNOWN } } }
I20260923 22:10:55.910570 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.908642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.1.161.131:41103)
0923 22:10:55.908676 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:10:55.908678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:10:55.908686 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:10:55.908745 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:10:55.908746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:10:55.908764 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:10:55.908824 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:10:55.908937 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.909307 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.909923 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:10:55.910083 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:10:55.910152 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:10:55.910283 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:10:55.910366 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:10:55.910444 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:10:55.910477 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 22:10:55.910494 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:10:55.911962 17828 raft_consensus.cc:1261] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Refusing update from remote peer 7519c85f99014678b012510539c95ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:55.911975 17674 raft_consensus.cc:1261] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Refusing update from remote peer 7519c85f99014678b012510539c95ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:10:55.912281 18021 consensus_queue.cc:1035] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:55.912420 18023 consensus_queue.cc:1035] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:10:55.914829 18027 mvcc.cc:205] Tried to move back new op lower bound from 7332665163413454848 to 7332665163320053760. Current Snapshot: MvccSnapshot[applied={T|T < 7332665163413454848}]
I20260923 22:10:55.915098 18029 mvcc.cc:205] Tried to move back new op lower bound from 7332665163413454848 to 7332665163320053760. Current Snapshot: MvccSnapshot[applied={T|T < 7332665163413454848}]
I20260923 22:10:55.915482 18026 mvcc.cc:205] Tried to move back new op lower bound from 7332665163413454848 to 7332665163320053760. Current Snapshot: MvccSnapshot[applied={T|T < 7332665163413454848 or (T in {7332665163413454848})}]
I20260923 22:10:55.916461 17548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:55.916548 17548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:10:55.917133 17548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Sending DeleteTablet for 3 replicas of tablet 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:10:55.917418 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17587
I20260923 22:10:55.917420 17940 tablet_service.cc:1514] Processing DeleteTablet for tablet 17371d1ab7d740f7895291b07dad7fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:55 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38852
I20260923 22:10:55.917448 17808 tablet_service.cc:1514] Processing DeleteTablet for tablet 17371d1ab7d740f7895291b07dad7fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:10:55 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:48652
I20260923 22:10:55.917598 18035 tablet_replica.cc:330] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: stopping tablet replica
I20260923 22:10:55.917662 18035 raft_consensus.cc:2229] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:10:55.917723 18035 raft_consensus.cc:2258] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:55.918043 18036 tablet_replica.cc:330] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: stopping tablet replica
I20260923 22:10:55.918118 18036 raft_consensus.cc:2229] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:10:55.918195 18036 raft_consensus.cc:2258] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:10:55.918267 18035 ts_tablet_manager.cc:1905] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:55.918530 18036 ts_tablet_manager.cc:1905] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:10:55.919922 18036 ts_tablet_manager.cc:1918] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:55.919940 18035 ts_tablet_manager.cc:1918] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:10:55.919977 18036 log.cc:1196] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/17371d1ab7d740f7895291b07dad7fc2
I20260923 22:10:55.919984 18035 log.cc:1196] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/17371d1ab7d740f7895291b07dad7fc2
I20260923 22:10:55.920220 18036 ts_tablet_manager.cc:1939] T 17371d1ab7d740f7895291b07dad7fc2 P 9287851cf1d048398c74fa4640a6a95f: Deleting consensus metadata
I20260923 22:10:55.920228 18035 ts_tablet_manager.cc:1939] T 17371d1ab7d740f7895291b07dad7fc2 P 7519c85f99014678b012510539c95ba6: Deleting consensus metadata
I20260923 22:10:55.920565 17535 catalog_manager.cc:4841] TS 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): tablet 17371d1ab7d740f7895291b07dad7fc2 (table test-table [id=329f796d616042c7b17d366cf156e6b1]) successfully deleted
I20260923 22:10:55.920572 17533 catalog_manager.cc:4841] TS 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): tablet 17371d1ab7d740f7895291b07dad7fc2 (table test-table [id=329f796d616042c7b17d366cf156e6b1]) successfully deleted
W20260923 22:10:55.924074 17532 connection.cc:535] client connection to 127.1.161.129:46725 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 22:10:55.924170 17532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:10:55.924623 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:10:55.924515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46725 (local address 127.0.0.1:48992)
0923 22:10:55.924564 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:10:55.924602 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:10:55.924610 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17741
W20260923 22:10:55.925738 17532 catalog_manager.cc:4584] TS d60ab4b2740844ffba07e609a38ed085 (127.1.161.129:46725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17371d1ab7d740f7895291b07dad7fc2: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46725: connect: Connection refused (error 111)
I20260923 22:10:55.929210 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17877
I20260923 22:10:55.933490 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 17515
I20260923 22:11:15.938687 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--webserver_interface=127.1.161.190
--webserver_port=46827
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:16.009642 18038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:16.009842 18038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:16.009877 18038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:16.010890 18038 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 22:11:16.010948 18038 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 22:11:16.010968 18038 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 22:11:16.010986 18038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:16.011015 18038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:16.011037 18038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:16.011060 18038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:16.011087 18038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:16.012413 18038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:16.012653 18038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:16.012846 18038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:16.017817 18044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.017856 18038 server_base.cc:1034] running on GCE node
W20260923 22:11:16.017831 18046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:16.017838 18043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.018231 18038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:16.018499 18038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:16.019657 18038 hybrid_clock.cc:648] HybridClock initialized: now 1790201476019643 us; error 42 us; skew 500 ppm
Sep 23 22:11:16 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:16.021792 18038 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:16.022186 18038 webserver.cc:460] Webserver started at http://127.1.161.190:46827/ using document root <none> and password file <none>
I20260923 22:11:16.022401 18038 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:16.022464 18038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:16.023782 18038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:16.024384 18053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.024540 18038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:16.024631 18038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "69130788682a495a9b8db93c465f1225"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "20d4fb9114c78666074caebb76607c90"
server_key_iv: "d94f86f3c802db026276619ac43f2a25"
server_key_version: "encryptionkey@0"
I20260923 22:11:16.024919 18038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:16.032335 18038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:16.033896 18038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:16.034026 18038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:16.037539 18038 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:35087
I20260923 22:11:16.037603 18105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:35087 every 8 connection(s)
I20260923 22:11:16.037945 18038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:16.039142 18106 sys_catalog.cc:263] Verifying existing consensus state
I20260923 22:11:16.039769 18106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap starting.
I20260923 22:11:16.043154 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18038
I20260923 22:11:16.043439 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:46725
--local_ip_for_outbound_sockets=127.1.161.129
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=46663
--webserver_interface=127.1.161.129
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:11:16.047626 18106 log.cc:824] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:16.049387 18106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:11:16.049655 18106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Bootstrap complete.
I20260923 22:11:16.051518 18106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:11:16.051695 18106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69130788682a495a9b8db93c465f1225, State: Initialized, Role: FOLLOWER
I20260923 22:11:16.051833 18106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:11:16.051903 18106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:16.051945 18106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:16.051988 18106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 2 FOLLOWER]: Advancing to term 3
I20260923 22:11:16.052834 18106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:11:16.052927 18106 leader_election.cc:302] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69130788682a495a9b8db93c465f1225; no voters:
I20260923 22:11:16.053092 18106 leader_election.cc:288] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 22:11:16.053123 18110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 22:11:16.053263 18110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [term 3 LEADER]: Becoming Leader. State: Replica: 69130788682a495a9b8db93c465f1225, State: Running, Role: LEADER
I20260923 22:11:16.053316 18106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:16.053386 18110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } }
I20260923 22:11:16.053617 18110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69130788682a495a9b8db93c465f1225. Latest consensus state: current_term: 3 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:11:16.053619 18112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "69130788682a495a9b8db93c465f1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69130788682a495a9b8db93c465f1225" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 35087 } } }
I20260923 22:11:16.053773 18110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:16.053781 18112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:16.054009 18119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:16.054847 18119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:16.055080 18119 catalog_manager.cc:1259] Loaded cluster ID: 04f8f7b986654dc78147e23c8fb4798c
I20260923 22:11:16.055265 18119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:16.055990 18119 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:16.056175 18119 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Loaded TSK: 0
I20260923 22:11:16.056232 18119 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69130788682a495a9b8db93c465f1225: Loaded TSK: 1
F20260923 22:11:16.056453 18119 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f705ad159ad google::LogMessage::Fail() at ??:0
@ 0x7f705ad1888a google::LogMessage::SendToLog() at ??:0
@ 0x7f705ad154de google::LogMessage::Flush() at ??:0
@ 0x7f705ad16a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f705d482104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f705d47fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f7062c731cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f7062c7348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7062c73594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f7062c39c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7062c43ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f705cbe30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f705cbd8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f705da3c6db start_thread at ??:0
@ 0x7f705981361f clone at ??:0
W20260923 22:11:16.131718 18108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:16.131875 18108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:16.131896 18108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:16.132843 18108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:16.132886 18108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:16.132910 18108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:16.134135 18108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:46725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=46663
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:16.134315 18108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:16.134509 18108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:16.138897 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:16.138934 18147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:16.138934 18148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.139101 18108 server_base.cc:1034] running on GCE node
I20260923 22:11:16.139369 18108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:16.139587 18108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:16.140735 18108 hybrid_clock.cc:648] HybridClock initialized: now 1790201476140716 us; error 38 us; skew 500 ppm
Sep 23 22:11:16 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:16.142993 18108 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:11:16.143460 18108 webserver.cc:460] Webserver started at http://127.1.161.129:46663/ using document root <none> and password file <none>
I20260923 22:11:16.143654 18108 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:16.143714 18108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:16.144899 18108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:16.145582 18157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.145746 18108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:16.145828 18108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "d60ab4b2740844ffba07e609a38ed085"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7b04d903da893b47521160f963e0cd09"
server_key_iv: "0a69ee285f967a78f4730f2ebf64afe5"
server_key_version: "encryptionkey@0"
I20260923 22:11:16.146128 18108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:16.172708 18108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:16.174691 18108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:16.174842 18108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:16.175050 18108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:16.175514 18164 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 22:11:16.177325 18108 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 22:11:16.177388 18108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 22:11:16.177450 18108 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 22:11:16.178007 18108 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 22:11:16.178061 18108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 22:11:16.178181 18164 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Bootstrap starting.
I20260923 22:11:16.184499 18108 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46725
I20260923 22:11:16.184559 18271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46725 every 8 connection(s)
I20260923 22:11:16.184823 18108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:16.188308 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18108
I20260923 22:11:16.188549 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:36321
--local_ip_for_outbound_sockets=127.1.161.130
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=46713
--webserver_interface=127.1.161.130
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:11:16.188697 18164 log.cc:824] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:16.190979 18164 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 22:11:16.191282 18164 tablet_bootstrap.cc:492] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Bootstrap complete.
I20260923 22:11:16.192328 18164 ts_tablet_manager.cc:1397] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.010s
I20260923 22:11:16.193490 18164 raft_consensus.cc:348] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:16.193763 18164 raft_consensus.cc:729] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ab4b2740844ffba07e609a38ed085, State: Initialized, Role: FOLLOWER
I20260923 22:11:16.193908 18164 consensus_queue.cc:260] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:16.194142 18164 ts_tablet_manager.cc:1428] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260923 22:11:16.223600 18274 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:11:16.185392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:36371)
0923 22:11:16.185551 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.185579 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:16.185951 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:16.223428 (+ 37477us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260923 22:11:16.223837 18272 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:35087 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:35087: Client connection negotiation failed: client connection to 127.1.161.190:35087: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:11:16.224160 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.224057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:33851)
0923 22:11:16.224107 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.224144 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 22:11:16.224465 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.224396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:41053)
0923 22:11:16.224432 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.224452 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 22:11:16.224570 18272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 22:11:16.263849 18276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:16.264011 18276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:16.264031 18276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:16.265039 18276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:16.265074 18276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:16.265118 18276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:16.266402 18276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:36321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=46713
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:16.266623 18276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:16.266831 18276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:16.271181 18285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:16.271202 18282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.271265 18276 server_base.cc:1034] running on GCE node
W20260923 22:11:16.271270 18283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.271685 18276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:16.271906 18276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:16.273062 18276 hybrid_clock.cc:648] HybridClock initialized: now 1790201476273050 us; error 40 us; skew 500 ppm
Sep 23 22:11:16 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:16.275111 18276 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:11:16.275499 18276 webserver.cc:460] Webserver started at http://127.1.161.130:46713/ using document root <none> and password file <none>
I20260923 22:11:16.275709 18276 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:16.275769 18276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:16.277019 18276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:16.277561 18292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.277690 18276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:16.277773 18276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "9287851cf1d048398c74fa4640a6a95f"
format_stamp: "Formatted at 2026-09-23 22:10:34 on dist-test-slave-npvd"
server_key: "7771a4f3e36279a0a5ba8849c57e1536"
server_key_iv: "404e570014c7764e9f72f3f35be28374"
server_key_version: "encryptionkey@0"
I20260923 22:11:16.278085 18276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:16.290799 18276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:16.292300 18276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:16.292457 18276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:16.292645 18276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:16.292914 18276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:16.292959 18276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.292987 18276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:16.293009 18276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.298385 18276 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:36321
I20260923 22:11:16.298415 18405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:36321 every 8 connection(s)
I20260923 22:11:16.298697 18276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:16.299571 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.299283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:38197)
0923 22:11:16.299406 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.299460 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
W20260923 22:11:16.299904 18406 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:35087 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:35087: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
I20260923 22:11:16.300132 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.300058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:43483)
0923 22:11:16.300094 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.300117 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:11:16.300357 18408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.300293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:37527)
0923 22:11:16.300327 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.300345 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260923 22:11:16.300472 18406 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:11:16.303562 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18276
I20260923 22:11:16.303843 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:41103
--local_ip_for_outbound_sockets=127.1.161.131
--tserver_master_addrs=127.1.161.190:35087
--webserver_port=35785
--webserver_interface=127.1.161.131
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:16.376403 18409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:16.376583 18409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:16.376605 18409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:16.377692 18409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:16.377727 18409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:16.377750 18409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:16.378985 18409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:41103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=35785
--tserver_master_addrs=127.1.161.190:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:16.379128 18409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:16.379339 18409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:16.383747 18415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:16.383756 18414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.383771 18409 server_base.cc:1034] running on GCE node
W20260923 22:11:16.383756 18417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:16.384104 18409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:16.384263 18409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:16.385411 18409 hybrid_clock.cc:648] HybridClock initialized: now 1790201476385393 us; error 29 us; skew 500 ppm
Sep 23 22:11:16 dist-test-slave-npvd krb5kdc[16970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:16.387734 18409 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:11:16.388098 18409 webserver.cc:460] Webserver started at http://127.1.161.131:35785/ using document root <none> and password file <none>
I20260923 22:11:16.388242 18409 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:16.388276 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:16.389276 18409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:16.389712 18424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.389842 18409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.389925 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "7519c85f99014678b012510539c95ba6"
format_stamp: "Formatted at 2026-09-23 22:10:35 on dist-test-slave-npvd"
server_key: "e3c314ab85345d1050919ea6456de8ad"
server_key_iv: "07dc9ae86664b17dd1611d95c33a619b"
server_key_version: "encryptionkey@0"
I20260923 22:11:16.390151 18409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:16.396535 18409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:16.398155 18409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:16.398253 18409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:16.398387 18409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:16.398617 18409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:16.398646 18409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.398663 18409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:16.398674 18409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:16.404964 18409 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:41103
I20260923 22:11:16.405019 18537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:41103 every 8 connection(s)
I20260923 22:11:16.405324 18409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:16.406250 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.405869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:56109)
0923 22:11:16.406023 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.406097 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260923 22:11:16.406584 18538 heartbeater.cc:644] Failed to heartbeat to 127.1.161.190:35087 (0 consecutive failures): Network error: Failed to ping master at 127.1.161.190:35087: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
I20260923 22:11:16.406885 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.406785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:43283)
0923 22:11:16.406838 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.406868 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:11:16.407143 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:16.407060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:44319)
0923 22:11:16.407102 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:16.407124 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:11:16.407266 18538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 22:11:16.407886 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18409
I20260923 22:11:17.225032 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.224839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:44835)
0923 22:11:17.224957 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.224987 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260923 22:11:17.300894 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.300723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:38673)
0923 22:11:17.300818 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.300853 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":48,"threads_started":1}
I20260923 22:11:17.407729 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.407526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:54493)
0923 22:11:17.407661 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.407688 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260923 22:11:17.529066 18545 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:17.529175 18545 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:17.529503 18545 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
Sep 23 22:11:17 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.131@KRBTEST.COM
Sep 23 22:11:17 dist-test-slave-npvd krb5kdc[16970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.130@KRBTEST.COM
I20260923 22:11:17.538800 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.529699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:47047 (local address 127.1.161.130:36321)
0923 22:11:17.529843 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:17.529857 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:17.529881 (+ 24us) server_negotiation.cc:407] Connection header received
0923 22:11:17.530258 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:17.530260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:17.530293 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:17.530433 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:17.531117 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.531921 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.532809 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.533188 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.533291 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:17.535757 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:17.535760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:17.535761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:17.535769 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:17.537554 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537846 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:17.537848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:17.537848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:17.537917 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:17.538088 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:17.538089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:17.538089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:17.538332 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:17.538400 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:17.538590 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 22:11:17.538703 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":47,"threads_started":1}
I20260923 22:11:17.538800 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.529734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:56485 (local address 127.1.161.131:41103)
0923 22:11:17.529834 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:17.529845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:17.529883 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:11:17.530256 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:17.530258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:17.530293 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:17.530416 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:17.531099 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.531923 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.532806 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.533186 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.533287 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:17.535386 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:17.535389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:17.535389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:17.535397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:17.537294 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537652 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:17.537654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:17.537655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:17.537726 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537957 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:17.537958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:17.537958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:17.538239 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:17.538312 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:17.538590 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 22:11:17.538704 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":41,"threads_started":1}
I20260923 22:11:17.538806 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.529597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36321 (local address 127.1.161.129:47047)
0923 22:11:17.529644 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.529653 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:17.529702 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:17.530453 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:17.530455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:17.530813 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:17.530985 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:17.530990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.532020 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.532021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:17.532729 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:17.532731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.533266 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.533267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:17.533372 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:17.533890 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:17.533897 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:17.535666 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:17.537619 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:17.537620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:17.537778 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:17.537993 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:17.537994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:17.538035 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:17.538385 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:17.538386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:17.538420 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 22:11:17.538533 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:11:17.538688 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":40}
I20260923 22:11:17.538852 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:17.529629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41103 (local address 127.1.161.129:56485)
0923 22:11:17.529723 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:17.529733 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:17.529776 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:17.530453 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:17.530455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:17.530816 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:17.530985 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:17.530990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.532016 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.532017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:17.532728 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:17.532730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:17.533264 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:17.533265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:17.533368 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:17.533953 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:17.533957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:17.535299 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:17.537375 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:17.537381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:17.537561 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:17.537781 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:17.537781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:17.537781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:17.537901 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:17.538298 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:17.538303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:17.538420 (+ 117us) client_negotiation.cc:769] Sending connection context
0923 22:11:17.538533 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:11:17.538734 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":123,"thread_start_us":40,"threads_started":1}
W20260923 22:11:17.539572 18161 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
W20260923 22:11:17.540009 18159 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:11:17.540060 18159 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ab4b2740844ffba07e609a38ed085; no voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f
I20260923 22:11:17.540151 18545 raft_consensus.cc:2726] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:18.225625 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.225376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:36175)
0923 22:11:18.225556 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.225588 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":51,"threads_started":1}
I20260923 22:11:18.301378 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.301220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:58937)
0923 22:11:18.301316 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.301346 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":50,"threads_started":1}
I20260923 22:11:18.408186 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:18.408028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:33395)
0923 22:11:18.408124 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:18.408155 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
I20260923 22:11:19.199019 18552 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:19.199105 18552 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:19.199410 18552 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
W20260923 22:11:19.199590 18159 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
W20260923 22:11:19.199695 18161 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:11:19.199734 18161 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ab4b2740844ffba07e609a38ed085; no voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f
I20260923 22:11:19.199815 18552 raft_consensus.cc:2726] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:19.226387 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.226096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:35973)
0923 22:11:19.226231 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.226342 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260923 22:11:19.301970 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.301773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:49567)
0923 22:11:19.301904 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.301936 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260923 22:11:19.408680 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:19.408510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:60321)
0923 22:11:19.408620 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:19.408647 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260923 22:11:20.227074 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.226857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:56107)
0923 22:11:20.226996 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.227029 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260923 22:11:20.302511 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.302322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:43399)
0923 22:11:20.302443 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.302477 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":50,"threads_started":1}
I20260923 22:11:20.409318 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:20.409152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:50405)
0923 22:11:20.409256 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:20.409286 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
I20260923 22:11:20.932861 18559 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:20.932955 18559 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:20.933121 18559 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
W20260923 22:11:20.933377 18159 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
W20260923 22:11:20.933456 18161 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:11:20.933512 18161 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ab4b2740844ffba07e609a38ed085; no voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f
I20260923 22:11:20.933588 18559 raft_consensus.cc:2726] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:21.227675 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.227499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:50705)
0923 22:11:21.227604 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.227635 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260923 22:11:21.303210 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.302929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:59081)
0923 22:11:21.303046 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.303101 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":50,"threads_started":1}
I20260923 22:11:21.409924 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:21.409770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:50051)
0923 22:11:21.409866 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:21.409894 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
I20260923 22:11:22.228252 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.228051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:37135)
0923 22:11:22.228174 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.228209 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260923 22:11:22.303755 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.303574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:45349)
0923 22:11:22.303686 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.303718 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260923 22:11:22.410421 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:22.410260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:58639)
0923 22:11:22.410354 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:22.410392 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260923 22:11:23.228919 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.228683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:57901)
0923 22:11:23.228833 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.228874 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":57,"threads_started":1}
I20260923 22:11:23.304283 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.304107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:38755)
0923 22:11:23.304222 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.304250 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":49,"threads_started":1}
I20260923 22:11:23.411032 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:23.410862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:51433)
0923 22:11:23.410975 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:23.411004 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260923 22:11:23.809257 18569 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:23.809350 18569 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:23.809521 18569 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
W20260923 22:11:23.809837 18161 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
W20260923 22:11:23.809975 18159 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:11:23.810019 18159 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ab4b2740844ffba07e609a38ed085; no voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f
I20260923 22:11:23.810111 18569 raft_consensus.cc:2726] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:24.229524 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.229336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:59313)
0923 22:11:24.229448 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.229480 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260923 22:11:24.304986 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.304650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:43859)
0923 22:11:24.304910 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.304940 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":79,"threads_started":1}
I20260923 22:11:24.411598 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:24.411426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:50703)
0923 22:11:24.411534 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:24.411566 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
I20260923 22:11:25.230099 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.229922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:46401)
0923 22:11:25.230032 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.230061 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260923 22:11:25.305533 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.305375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:48189)
0923 22:11:25.305478 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.305504 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260923 22:11:25.412158 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:25.411973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:51951)
0923 22:11:25.412098 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:25.412128 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260923 22:11:26.230621 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.230438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:48203)
0923 22:11:26.230549 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.230580 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":50,"threads_started":1}
I20260923 22:11:26.306090 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.305932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:60099)
0923 22:11:26.306027 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.306057 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":48,"threads_started":1}
I20260923 22:11:26.412679 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:26.412522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:35987)
0923 22:11:26.412620 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:26.412648 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260923 22:11:27.231212 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.230979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:35863)
0923 22:11:27.231101 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.231132 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":52,"threads_started":1}
I20260923 22:11:27.306599 18580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.306429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:38923)
0923 22:11:27.306532 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.306563 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260923 22:11:27.413197 18581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:27.413011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:59195)
0923 22:11:27.413137 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:27.413164 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260923 22:11:28.231930 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.231755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:45055)
0923 22:11:28.231857 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.231885 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
I20260923 22:11:28.307159 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.306976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:51963)
0923 22:11:28.307082 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.307108 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
I20260923 22:11:28.413741 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:28.413561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:53105)
0923 22:11:28.413677 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:28.413704 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260923 22:11:29.154997 18585 raft_consensus.cc:482] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:29.155078 18585 raft_consensus.cc:504] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9287851cf1d048398c74fa4640a6a95f" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36321 } } peers { permanent_uuid: "7519c85f99014678b012510539c95ba6" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41103 } } peers { permanent_uuid: "d60ab4b2740844ffba07e609a38ed085" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46725 } }
I20260923 22:11:29.155303 18585 leader_election.cc:288] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321), 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103)
W20260923 22:11:29.155616 18159 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9287851cf1d048398c74fa4640a6a95f (127.1.161.130:36321): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
W20260923 22:11:29.155661 18161 leader_election.cc:341] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7519c85f99014678b012510539c95ba6 (127.1.161.131:41103): Not found: Tablet not found: 17371d1ab7d740f7895291b07dad7fc2
I20260923 22:11:29.155704 18159 leader_election.cc:302] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60ab4b2740844ffba07e609a38ed085; no voters: 7519c85f99014678b012510539c95ba6, 9287851cf1d048398c74fa4640a6a95f
I20260923 22:11:29.155772 18585 raft_consensus.cc:2726] T 17371d1ab7d740f7895291b07dad7fc2 P d60ab4b2740844ffba07e609a38ed085 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 22:11:29.232479 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.232301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:56281)
0923 22:11:29.232415 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.232445 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
I20260923 22:11:29.307909 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.307582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:52485)
0923 22:11:29.307839 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.307871 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":210,"threads_started":1}
I20260923 22:11:29.414327 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:29.414131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:39977)
0923 22:11:29.414258 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:29.414285 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":51,"threads_started":1}
I20260923 22:11:30.233045 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.232866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:40367)
0923 22:11:30.232979 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:30.233009 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
I20260923 22:11:30.308513 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.308284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:54769)
0923 22:11:30.308431 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:30.308467 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260923 22:11:30.414968 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:30.414786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:42313)
0923 22:11:30.414911 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:30.414940 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260923 22:11:31.233711 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.233535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.129:50529)
0923 22:11:31.233638 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.233674 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260923 22:11:31.309196 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.309026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.130:44443)
0923 22:11:31.309133 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.309160 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260923 22:11:31.415570 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:31.415402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:35087 (local address 127.1.161.131:43123)
0923 22:11:31.415515 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:31.415542 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.190:35087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260923 22:11:31.416204 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18108
I20260923 22:11:31.420344 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18276
I20260923 22:11:31.424690 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18409
2026-09-23T22:11:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58045 ms)
[----------] 30 tests from SecurityITest (216784 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:31 dist-test-slave-npvd krb5kdc[18597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:31 dist-test-slave-npvd krb5kdc[18597](info): set up 2 sockets
Sep 23 22:11:31 dist-test-slave-npvd krb5kdc[18597](info): commencing operation
krb5kdc: starting...
W20260923 22:11:32.354581 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.911s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:32.461086 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:43609
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:32.530931 18610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:32.531141 18610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:32.531179 18610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:32.532200 18610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:32.532259 18610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:32.532282 18610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:32.532307 18610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:32.532335 18610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:32.533598 18610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:32.533830 18610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:32.534035 18610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:32.538281 18618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.538316 18615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.538297 18616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:32.538327 18610 server_base.cc:1034] running on GCE node
I20260923 22:11:32.538751 18610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:32.539016 18610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:32.540170 18610 hybrid_clock.cc:648] HybridClock initialized: now 1790201492540159 us; error 39 us; skew 500 ppm
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:32.542204 18610 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:32.542559 18610 webserver.cc:460] Webserver started at http://127.1.161.190:37335/ using document root <none> and password file <none>
I20260923 22:11:32.542774 18610 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:32.542855 18610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:32.542987 18610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:32.543879 18610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "7241b4a64415481881ded625043bbbcf"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "aa9692908ecfac3881d99ade92158916"
server_key_iv: "ff700937db59ee29023fdea254c52e03"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.544183 18610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "7241b4a64415481881ded625043bbbcf"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "aa9692908ecfac3881d99ade92158916"
server_key_iv: "ff700937db59ee29023fdea254c52e03"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.545349 18610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:32.545957 18625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.546120 18610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:32.546211 18610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "7241b4a64415481881ded625043bbbcf"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "aa9692908ecfac3881d99ade92158916"
server_key_iv: "ff700937db59ee29023fdea254c52e03"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.546283 18610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:32.574203 18610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:32.575875 18610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:32.576026 18610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:32.579711 18610 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:43609
I20260923 22:11:32.579741 18677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:43609 every 8 connection(s)
I20260923 22:11:32.580027 18610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:32.580782 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:32.582646 18678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: Bootstrap starting.
I20260923 22:11:32.583117 18678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:32.583384 18678 log.cc:824] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:32.583987 18678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: No bootstrap required, opened a new log
I20260923 22:11:32.584856 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18610
I20260923 22:11:32.584931 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:32.585105 1670 external_mini_cluster.cc:1428] Setting key 80bcb8baa4e58612abf3b0f4b83fa33c
I20260923 22:11:32.585645 18678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } }
I20260923 22:11:32.585793 18678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:32.585822 18678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7241b4a64415481881ded625043bbbcf, State: Initialized, Role: FOLLOWER
I20260923 22:11:32.585924 18678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } }
I20260923 22:11:32.585997 18678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:32.586025 18678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:32.586110 18678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:32.586665 18678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } }
I20260923 22:11:32.586753 18678 leader_election.cc:302] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7241b4a64415481881ded625043bbbcf; no voters:
I20260923 22:11:32.586874 18678 leader_election.cc:288] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:32.586942 18683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:32.587071 18678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:32.587086 18683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [term 1 LEADER]: Becoming Leader. State: Replica: 7241b4a64415481881ded625043bbbcf, State: Running, Role: LEADER
I20260923 22:11:32.587160 18683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } }
I20260923 22:11:32.587568 18683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7241b4a64415481881ded625043bbbcf. Latest consensus state: current_term: 1 leader_uuid: "7241b4a64415481881ded625043bbbcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } } }
I20260923 22:11:32.587594 18684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7241b4a64415481881ded625043bbbcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7241b4a64415481881ded625043bbbcf" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 43609 } } }
I20260923 22:11:32.587733 18684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:32.587697 18683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:32.587946 18692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:32.589363 18692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:32.590662 18692 catalog_manager.cc:1347] Generated new cluster ID: 531c370911d9475ebb8dc3f407c06859
I20260923 22:11:32.590709 18692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:32.594347 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.585521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.1.161.190:43609)
0923 22:11:32.585708 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.585716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.585742 (+ 26us) server_negotiation.cc:407] Connection header received
0923 22:11:32.585835 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.585840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.585884 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.586002 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:32.586594 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.587975 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.589007 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.589342 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.589452 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.591509 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:32.591512 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:32.591512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:32.591523 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:32.593032 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.593371 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.593374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.593374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.593475 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.593667 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.593668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.593668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.593854 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:32.593999 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.594055 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.594170 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260923 22:11:32.610137 18692 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:32.610721 18692 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:32.622076 18692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: Generated new TSK 0
I20260923 22:11:32.622222 18692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:11:32.650578 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:32.722995 18704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:32.723201 18704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:32.723233 18704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:32.724223 18704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:32.724260 18704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:32.724282 18704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:32.725418 18704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:32.725567 18704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:32.725700 18704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:32.730032 18709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.730032 18712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.730031 18710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:32.730108 18704 server_base.cc:1034] running on GCE node
I20260923 22:11:32.730353 18704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:32.730546 18704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:32.731681 18704 hybrid_clock.cc:648] HybridClock initialized: now 1790201492731681 us; error 35 us; skew 500 ppm
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:32.733858 18704 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:11:32.734279 18704 webserver.cc:460] Webserver started at http://127.1.161.129:37005/ using document root <none> and password file <none>
I20260923 22:11:32.734468 18704 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:32.734525 18704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:32.734639 18704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:32.735526 18704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "42a3d73db92541f49f8f3d911a1d9b29"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "1dabff96df7c6380e181cf83ad6deb21"
server_key_iv: "f687a6f4a92f96045fbebab66c10852a"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.735841 18704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "42a3d73db92541f49f8f3d911a1d9b29"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "1dabff96df7c6380e181cf83ad6deb21"
server_key_iv: "f687a6f4a92f96045fbebab66c10852a"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.736967 18704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:32.737612 18719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.737785 18704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:32.737877 18704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "42a3d73db92541f49f8f3d911a1d9b29"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "1dabff96df7c6380e181cf83ad6deb21"
server_key_iv: "f687a6f4a92f96045fbebab66c10852a"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.737959 18704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:32.755728 18704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:32.757325 18704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:32.757483 18704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:32.757700 18704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:32.757982 18704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:32.758033 18704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.758083 18704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:32.758111 18704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.763255 18704 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:37533
I20260923 22:11:32.763340 18832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:37533 every 8 connection(s)
I20260923 22:11:32.763609 18704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:32.764235 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18704
I20260923 22:11:32.764353 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:32.764513 1670 external_mini_cluster.cc:1428] Setting key 3781d5bcf55649aacbabe5a98747c10b
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
I20260923 22:11:32.773829 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.764286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:56989 (local address 127.1.161.190:43609)
0923 22:11:32.764443 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.764445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.764485 (+ 40us) server_negotiation.cc:407] Connection header received
0923 22:11:32.765134 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.765135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.765150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.765209 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:32.767067 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.767564 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.769050 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.769226 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.769270 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.771348 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:32.771350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:32.771350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:32.771356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:32.772609 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.772968 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.772969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.772969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.773018 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.773244 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.773244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.773245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.773356 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:32.773417 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.773677 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.773750 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260923 22:11:32.774005 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.764224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:43609 (local address 127.1.161.129:56989)
0923 22:11:32.764385 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.764418 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.765003 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.766319 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.766322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.766766 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:32.766973 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.766979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.767652 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.767653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.768964 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.768968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.769301 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.769302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.769434 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.769827 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:32.769836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:32.771268 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:32.772696 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:32.772702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:32.772704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:32.772893 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:32.773080 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:32.773081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:32.773081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:32.773173 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:32.773419 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:32.773426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:32.773545 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.773654 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.773786 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
I20260923 22:11:32.774466 18833 heartbeater.cc:347] Connected to a master server at 127.1.161.190:43609
I20260923 22:11:32.774561 18833 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:32.774734 18833 heartbeater.cc:511] Master 127.1.161.190:43609 requested a full tablet report, sending...
I20260923 22:11:32.775305 18642 ts_manager.cc:194] Registered new tserver with Master: 42a3d73db92541f49f8f3d911a1d9b29 (127.1.161.129:37533)
I20260923 22:11:32.776050 18642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:56989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:11:32.789776 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:32.858863 18838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:32.859009 18838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:32.859030 18838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:32.860018 18838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:32.860054 18838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:32.860100 18838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:32.861397 18838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:32.861617 18838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:32.861833 18838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:32.866411 18843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.866433 18844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:32.866410 18846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:32.866433 18838 server_base.cc:1034] running on GCE node
I20260923 22:11:32.866708 18838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:32.866937 18838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:32.868086 18838 hybrid_clock.cc:648] HybridClock initialized: now 1790201492868076 us; error 39 us; skew 500 ppm
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:32.870409 18838 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:11:32.870715 18838 webserver.cc:460] Webserver started at http://127.1.161.130:37083/ using document root <none> and password file <none>
I20260923 22:11:32.870872 18838 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:32.870904 18838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:32.870972 18838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:32.871783 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "2910c9034e564e4c8af365a8b2ccedb5"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "b48cd3bf4bc7753437a11ad348a1bf1e"
server_key_iv: "7765d79a754986e615d2bcf1d781071e"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.872021 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "2910c9034e564e4c8af365a8b2ccedb5"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "b48cd3bf4bc7753437a11ad348a1bf1e"
server_key_iv: "7765d79a754986e615d2bcf1d781071e"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.873050 18838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:32.873661 18853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.873797 18838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:32.873868 18838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "2910c9034e564e4c8af365a8b2ccedb5"
format_stamp: "Formatted at 2026-09-23 22:11:32 on dist-test-slave-npvd"
server_key: "b48cd3bf4bc7753437a11ad348a1bf1e"
server_key_iv: "7765d79a754986e615d2bcf1d781071e"
server_key_version: "encryptionkey@0"
I20260923 22:11:32.873914 18838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:32.897270 18838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:32.898856 18838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:32.898947 18838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:32.899096 18838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:32.899354 18838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:32.899384 18838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.899401 18838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:32.899412 18838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:32.904044 18838 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:38425
I20260923 22:11:32.904098 18966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:38425 every 8 connection(s)
I20260923 22:11:32.904348 18838 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:11:32 dist-test-slave-npvd krb5kdc[18597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201492, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:32.911836 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.904966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:57771 (local address 127.1.161.190:43609)
0923 22:11:32.905029 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:32.905031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:32.905129 (+ 98us) server_negotiation.cc:407] Connection header received
0923 22:11:32.905489 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:32.905490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:32.905498 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:32.905541 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:32.906280 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.906703 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.907613 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.907793 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.907845 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.909627 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:32.909628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:32.909628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:32.909632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:32.910604 (+ 972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.910978 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.910980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.910980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.911071 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:32.911314 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:32.911314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:32.911315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:32.911410 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:32.911486 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:32.911710 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 22:11:32.911779 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:11:32.911897 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:32.904925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:43609 (local address 127.1.161.130:57771)
0923 22:11:32.905050 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:32.905077 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:32.905420 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:32.905582 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:32.905585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:32.906059 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:32.906213 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.906217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.906771 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.906771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.907509 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:32.907510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:32.907863 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:32.907863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:32.907949 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:32.908409 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:32.908417 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:32.909563 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:32.910665 (+ 1102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:32.910670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:32.910671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:32.910843 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:32.911141 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:32.911142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:32.911142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:32.911259 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:32.911485 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:32.911489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:32.911605 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 22:11:32.911679 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 22:11:32.911774 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260923 22:11:32.912341 18967 heartbeater.cc:347] Connected to a master server at 127.1.161.190:43609
I20260923 22:11:32.912407 18967 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:32.912546 18967 heartbeater.cc:511] Master 127.1.161.190:43609 requested a full tablet report, sending...
I20260923 22:11:32.912868 18642 ts_manager.cc:194] Registered new tserver with Master: 2910c9034e564e4c8af365a8b2ccedb5 (127.1.161.130:38425)
I20260923 22:11:32.913215 18642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:57771
I20260923 22:11:32.914026 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18838
I20260923 22:11:32.914160 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:32.914355 1670 external_mini_cluster.cc:1428] Setting key 9ea6f99561ed5f1e1d8b30f9628b9534
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:11:32.937233 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:33.007642 18972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.007845 18972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.007882 18972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.008971 18972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.009029 18972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.009065 18972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:33.010253 18972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:33.010473 18972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.010660 18972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:11:33.015074 18972 server_base.cc:1034] running on GCE node
W20260923 22:11:33.015120 18978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.015199 18977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.015388 18980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.015590 18972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.015848 18972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.017046 18972 hybrid_clock.cc:648] HybridClock initialized: now 1790201493017025 us; error 32 us; skew 500 ppm
Sep 23 22:11:33 dist-test-slave-npvd krb5kdc[18597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:33.020237 18972 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:11:33.020675 18972 webserver.cc:460] Webserver started at http://127.1.161.131:44613/ using document root <none> and password file <none>
I20260923 22:11:33.020921 18972 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.020975 18972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.021100 18972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.022169 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "b5487ce2e8c5479b95855f06b2fb3895"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "1e567e43000fd6ed013e9d225d754110"
server_key_iv: "013dd5fcafec102df7be02648e59f5c3"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.022493 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "b5487ce2e8c5479b95855f06b2fb3895"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "1e567e43000fd6ed013e9d225d754110"
server_key_iv: "013dd5fcafec102df7be02648e59f5c3"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.023957 18972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:33.024567 18987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.024744 18972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:33.024828 18972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "b5487ce2e8c5479b95855f06b2fb3895"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "1e567e43000fd6ed013e9d225d754110"
server_key_iv: "013dd5fcafec102df7be02648e59f5c3"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.024905 18972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.036330 18972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.037902 18972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.038048 18972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.038301 18972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.038640 18972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.038697 18972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.038735 18972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.038772 18972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.044243 18972 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:46429
I20260923 22:11:33.044283 19100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:46429 every 8 connection(s)
I20260923 22:11:33.044533 18972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:11:33 dist-test-slave-npvd krb5kdc[18597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201493, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:33.051029 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 18972
I20260923 22:11:33.051134 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:33.051367 1670 external_mini_cluster.cc:1428] Setting key 347c54692a25fcc72b14b708775f6b3a
I20260923 22:11:33.052651 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.045138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:37325 (local address 127.1.161.190:43609)
0923 22:11:33.045174 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.045176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.045360 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:11:33.045860 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.045861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.045870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.045917 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.046698 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.047126 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.048085 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.048332 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.048381 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.050160 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.050161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.050162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:33.050168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.051471 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.051871 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.051872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.051872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.051925 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.052169 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.052169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.052169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.052299 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.052345 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.052537 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.052594 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:33.052781 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.045117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:43609 (local address 127.1.161.131:37325)
0923 22:11:33.045261 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.045295 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.045745 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.045987 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.045991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.046472 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.046632 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.046636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.047227 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.047227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.048009 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.048011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.048397 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.048397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.048491 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.048921 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:33.048927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.050068 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.051541 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.051548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.051549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.051805 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.051989 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:33.051990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:33.051990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:33.052110 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:33.052350 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.052354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.052442 (+ 88us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.052513 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.052609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260923 22:11:33.053223 19101 heartbeater.cc:347] Connected to a master server at 127.1.161.190:43609
I20260923 22:11:33.053292 19101 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.053399 19101 heartbeater.cc:511] Master 127.1.161.190:43609 requested a full tablet report, sending...
I20260923 22:11:33.053681 18642 ts_manager.cc:194] Registered new tserver with Master: b5487ce2e8c5479b95855f06b2fb3895 (127.1.161.131:46429)
I20260923 22:11:33.054001 18642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:37325
I20260923 22:11:33.063113 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:33.069203 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.064369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.1.161.190:43609)
0923 22:11:33.064425 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.064427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.064508 (+ 81us) server_negotiation.cc:407] Connection header received
0923 22:11:33.064540 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.064540 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.064547 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.064593 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.065039 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.065415 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.065995 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.066143 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.066185 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.067035 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.067035 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.067036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:33.067041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.068369 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.068664 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.068664 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.068664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.068710 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:33.068838 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:33.068838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:33.068839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:33.068948 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.069017 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.069100 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.069150 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:11:33.070477 18642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:33.072036 18642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:33.077770 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.073936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38425 (local address 127.0.0.1:53376)
0923 22:11:33.074491 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.074499 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.074546 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.074768 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.074769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.074770 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.074796 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.074797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075964 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.075965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.076700 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.076703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077434 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.077522 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077536 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.077652 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.077702 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"spinlock_wait_cycles":384,"thread_start_us":83,"threads_started":1}
I20260923 22:11:33.077790 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.074145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.1.161.130:38425)
0923 22:11:33.074373 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.074376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.074561 (+ 185us) server_negotiation.cc:407] Connection header received
0923 22:11:33.074609 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.074611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.074625 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.074689 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.074882 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075839 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.076787 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077338 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077429 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077489 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.077674 (+ 185us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.077711 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":70,"threads_started":1}
I20260923 22:11:33.078049 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.073931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37533 (local address 127.0.0.1:47584)
0923 22:11:33.074287 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.074323 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.074397 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.074662 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.074663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.074665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.074719 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.074720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075921 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.075921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.076859 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.076861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077796 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.077866 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077872 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.077918 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.077968 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":36,"spinlock_wait_cycles":768,"thread_start_us":57,"threads_started":1}
I20260923 22:11:33.078051 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.073932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:46429 (local address 127.0.0.1:39504)
0923 22:11:33.074288 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.074321 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.074408 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.074618 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.074620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.074622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.074671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.074672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075644 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.075645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.076751 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.076754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077805 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.077866 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077872 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.077918 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.077957 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":21,"thread_start_us":59,"threads_started":1}
I20260923 22:11:33.078310 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.074145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.1.161.131:46429)
0923 22:11:33.074302 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.074307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.074423 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:11:33.074467 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.074469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.074485 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.074561 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.074753 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075538 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.076935 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077648 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077698 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077748 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.078088 (+ 340us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.078236 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
I20260923 22:11:33.078392 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.074100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.1.161.129:37533)
0923 22:11:33.074270 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.074274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.074396 (+ 122us) server_negotiation.cc:407] Connection header received
0923 22:11:33.074481 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.074483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.074497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.074582 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.074783 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.075796 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.076953 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.077654 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.077716 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.077762 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.078260 (+ 498us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.078302 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260923 22:11:33.078665 18901 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 (DEFAULT_TABLE table=test-table [id=f7f59867ca894fba8c034f8f624f8efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.079008 18901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa3503dcb7f40a395fd536333ad2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.078912 19035 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 (DEFAULT_TABLE table=test-table [id=f7f59867ca894fba8c034f8f624f8efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.079077 18767 tablet_service.cc:1467] Processing CreateTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 (DEFAULT_TABLE table=test-table [id=f7f59867ca894fba8c034f8f624f8efc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.079223 19035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa3503dcb7f40a395fd536333ad2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.079357 18767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1aa3503dcb7f40a395fd536333ad2138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.081377 19115 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Bootstrap starting.
I20260923 22:11:33.081935 19116 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Bootstrap starting.
I20260923 22:11:33.081979 19115 tablet_bootstrap.cc:654] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.082146 19117 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Bootstrap starting.
I20260923 22:11:33.082540 19116 tablet_bootstrap.cc:654] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.082723 19117 tablet_bootstrap.cc:654] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.082875 19115 log.cc:824] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.083058 19116 log.cc:824] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.083251 19117 log.cc:824] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.083575 19115 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: No bootstrap required, opened a new log
I20260923 22:11:33.083669 19116 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: No bootstrap required, opened a new log
I20260923 22:11:33.083670 19115 ts_tablet_manager.cc:1397] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:33.083725 19116 ts_tablet_manager.cc:1397] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:11:33.083927 19117 tablet_bootstrap.cc:492] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: No bootstrap required, opened a new log
I20260923 22:11:33.084007 19117 ts_tablet_manager.cc:1397] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:11:33.085285 19117 raft_consensus.cc:348] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085419 19117 raft_consensus.cc:374] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.085455 19117 raft_consensus.cc:729] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a3d73db92541f49f8f3d911a1d9b29, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.085438 19115 raft_consensus.cc:348] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085438 19116 raft_consensus.cc:348] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085551 19117 consensus_queue.cc:260] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085592 19116 raft_consensus.cc:374] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.085592 19115 raft_consensus.cc:374] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.085625 19116 raft_consensus.cc:729] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2910c9034e564e4c8af365a8b2ccedb5, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.085635 19115 raft_consensus.cc:729] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5487ce2e8c5479b95855f06b2fb3895, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.085723 19115 consensus_queue.cc:260] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085732 19116 consensus_queue.cc:260] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.085769 19117 ts_tablet_manager.cc:1428] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:33.085861 18833 heartbeater.cc:503] Master 127.1.161.190:43609 was elected leader, sending a full tablet report...
I20260923 22:11:33.085952 19115 ts_tablet_manager.cc:1428] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:11:33.085952 19116 ts_tablet_manager.cc:1428] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:33.086078 19101 heartbeater.cc:503] Master 127.1.161.190:43609 was elected leader, sending a full tablet report...
I20260923 22:11:33.086274 18967 heartbeater.cc:503] Master 127.1.161.190:43609 was elected leader, sending a full tablet report...
W20260923 22:11:33.154857 18968 tablet.cc:2369] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:33.264286 18834 tablet.cc:2369] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:33.295059 19102 tablet.cc:2369] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:33.316665 19123 raft_consensus.cc:482] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:33.316762 19123 raft_consensus.cc:504] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.317193 19123 leader_election.cc:288] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42a3d73db92541f49f8f3d911a1d9b29 (127.1.161.129:37533), 2910c9034e564e4c8af365a8b2ccedb5 (127.1.161.130:38425)
I20260923 22:11:33.319981 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.317464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:40657 (local address 127.1.161.130:38425)
0923 22:11:33.317519 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.317521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.317635 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:11:33.317707 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.317708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.317718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.317789 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.317945 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.318400 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319276 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.319637 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319699 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.319707 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.319870 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.319922 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:11:33.319977 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.317349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37533 (local address 127.1.161.131:33715)
0923 22:11:33.317449 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.317459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.317518 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.317711 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.317712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.317713 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.317795 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.317797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.318443 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.318444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.319121 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.319122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.319651 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.319750 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.319753 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.319850 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.319913 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260923 22:11:33.320150 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.317366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:38425 (local address 127.1.161.131:40657)
0923 22:11:33.317551 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.317568 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.317650 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.317791 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.317791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.317793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.317871 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.317876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.318482 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.318483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.319164 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.319166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.319725 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.319775 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.319778 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.320051 (+ 273us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.320094 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260923 22:11:33.320151 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.317399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:33715 (local address 127.1.161.129:37533)
0923 22:11:33.317443 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.317447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.317596 (+ 149us) server_negotiation.cc:407] Connection header received
0923 22:11:33.317633 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.317634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.317642 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.317706 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.317867 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.318358 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319251 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.319564 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.319638 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.319647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.320061 (+ 414us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.320099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 22:11:33.320533 18921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa3503dcb7f40a395fd536333ad2138" candidate_uuid: "b5487ce2e8c5479b95855f06b2fb3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2910c9034e564e4c8af365a8b2ccedb5" is_pre_election: true
I20260923 22:11:33.320535 18787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa3503dcb7f40a395fd536333ad2138" candidate_uuid: "b5487ce2e8c5479b95855f06b2fb3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a3d73db92541f49f8f3d911a1d9b29" is_pre_election: true
I20260923 22:11:33.320672 18921 raft_consensus.cc:2454] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5487ce2e8c5479b95855f06b2fb3895 in term 0.
I20260923 22:11:33.320706 18787 raft_consensus.cc:2454] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5487ce2e8c5479b95855f06b2fb3895 in term 0.
I20260923 22:11:33.320894 18991 leader_election.cc:302] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2910c9034e564e4c8af365a8b2ccedb5, b5487ce2e8c5479b95855f06b2fb3895; no voters:
I20260923 22:11:33.321035 19123 raft_consensus.cc:2781] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:33.321099 19123 raft_consensus.cc:482] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:33.321130 19123 raft_consensus.cc:3037] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:33.321733 19123 raft_consensus.cc:504] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.321861 19123 leader_election.cc:288] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [CANDIDATE]: Term 1 election: Requested vote from peers 42a3d73db92541f49f8f3d911a1d9b29 (127.1.161.129:37533), 2910c9034e564e4c8af365a8b2ccedb5 (127.1.161.130:38425)
I20260923 22:11:33.321972 18787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa3503dcb7f40a395fd536333ad2138" candidate_uuid: "b5487ce2e8c5479b95855f06b2fb3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a3d73db92541f49f8f3d911a1d9b29"
I20260923 22:11:33.321990 18921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1aa3503dcb7f40a395fd536333ad2138" candidate_uuid: "b5487ce2e8c5479b95855f06b2fb3895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2910c9034e564e4c8af365a8b2ccedb5"
I20260923 22:11:33.322048 18921 raft_consensus.cc:3037] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:33.322054 18787 raft_consensus.cc:3037] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:33.322649 18921 raft_consensus.cc:2454] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5487ce2e8c5479b95855f06b2fb3895 in term 1.
I20260923 22:11:33.322739 18787 raft_consensus.cc:2454] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5487ce2e8c5479b95855f06b2fb3895 in term 1.
I20260923 22:11:33.322789 18991 leader_election.cc:302] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2910c9034e564e4c8af365a8b2ccedb5, b5487ce2e8c5479b95855f06b2fb3895; no voters:
I20260923 22:11:33.322878 19123 raft_consensus.cc:2781] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:33.323020 19123 raft_consensus.cc:686] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 1 LEADER]: Becoming Leader. State: Replica: b5487ce2e8c5479b95855f06b2fb3895, State: Running, Role: LEADER
I20260923 22:11:33.323107 19123 consensus_queue.cc:237] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } }
I20260923 22:11:33.323907 18642 catalog_manager.cc:5495] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5487ce2e8c5479b95855f06b2fb3895 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "b5487ce2e8c5479b95855f06b2fb3895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5487ce2e8c5479b95855f06b2fb3895" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 46429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 } health_report { overall_health: UNKNOWN } } }
I20260923 22:11:33.360401 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.358612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.1.161.131:46429)
0923 22:11:33.358688 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.358690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.358696 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:11:33.358731 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.358732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.358744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.358792 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:33.358889 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.359275 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.359878 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.360024 (+ 146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.360066 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.360194 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:33.360271 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:33.360321 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.360332 (+ 11us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.360345 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:11:33.361850 18921 raft_consensus.cc:1261] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 1 FOLLOWER]: Refusing update from remote peer b5487ce2e8c5479b95855f06b2fb3895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:33.361855 18787 raft_consensus.cc:1261] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 1 FOLLOWER]: Refusing update from remote peer b5487ce2e8c5479b95855f06b2fb3895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:33.362139 19125 consensus_queue.cc:1035] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a3d73db92541f49f8f3d911a1d9b29" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:33.362255 19123 consensus_queue.cc:1035] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2910c9034e564e4c8af365a8b2ccedb5" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:33.364185 19128 mvcc.cc:205] Tried to move back new op lower bound from 7332665316807733248 to 7332665316652175360. Current Snapshot: MvccSnapshot[applied={T|T < 7332665316807733248}]
I20260923 22:11:33.364320 19130 mvcc.cc:205] Tried to move back new op lower bound from 7332665316807733248 to 7332665316652175360. Current Snapshot: MvccSnapshot[applied={T|T < 7332665316807733248}]
I20260923 22:11:33.366070 18642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:33.366153 18642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:33.366765 18642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7241b4a64415481881ded625043bbbcf: Sending DeleteTablet for 3 replicas of tablet 1aa3503dcb7f40a395fd536333ad2138
I20260923 22:11:33.367025 19035 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:33 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:39504
I20260923 22:11:33.367031 18767 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:33 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:47584
I20260923 22:11:33.367049 18901 tablet_service.cc:1514] Processing DeleteTablet for tablet 1aa3503dcb7f40a395fd536333ad2138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:33 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:53376
I20260923 22:11:33.367249 19140 tablet_replica.cc:330] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: stopping tablet replica
I20260923 22:11:33.367311 19138 tablet_replica.cc:330] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: stopping tablet replica
I20260923 22:11:33.367389 19139 tablet_replica.cc:330] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29: stopping tablet replica
I20260923 22:11:33.367399 19138 raft_consensus.cc:2229] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:33.367455 19139 raft_consensus.cc:2229] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:33.367479 19138 raft_consensus.cc:2258] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:33.367558 19139 raft_consensus.cc:2258] T 1aa3503dcb7f40a395fd536333ad2138 P 42a3d73db92541f49f8f3d911a1d9b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:33.367646 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18704
I20260923 22:11:33.368314 19138 ts_tablet_manager.cc:1905] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:33.368870 19140 raft_consensus.cc:2229] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:33.369010 19140 raft_consensus.cc:2258] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:33.369405 19140 ts_tablet_manager.cc:1905] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:33.369907 19138 ts_tablet_manager.cc:1918] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:33.369958 19138 log.cc:1196] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/1aa3503dcb7f40a395fd536333ad2138
I20260923 22:11:33.370144 19138 ts_tablet_manager.cc:1939] T 1aa3503dcb7f40a395fd536333ad2138 P b5487ce2e8c5479b95855f06b2fb3895: Deleting consensus metadata
I20260923 22:11:33.370417 18628 catalog_manager.cc:4841] TS b5487ce2e8c5479b95855f06b2fb3895 (127.1.161.131:46429): tablet 1aa3503dcb7f40a395fd536333ad2138 (table test-table [id=f7f59867ca894fba8c034f8f624f8efc]) successfully deleted
I20260923 22:11:33.371224 19140 ts_tablet_manager.cc:1918] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:33.371287 19140 log.cc:1196] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/1aa3503dcb7f40a395fd536333ad2138
I20260923 22:11:33.371508 19140 ts_tablet_manager.cc:1939] T 1aa3503dcb7f40a395fd536333ad2138 P 2910c9034e564e4c8af365a8b2ccedb5: Deleting consensus metadata
I20260923 22:11:33.371848 18629 catalog_manager.cc:4841] TS 2910c9034e564e4c8af365a8b2ccedb5 (127.1.161.130:38425): tablet 1aa3503dcb7f40a395fd536333ad2138 (table test-table [id=f7f59867ca894fba8c034f8f624f8efc]) successfully deleted
W20260923 22:11:33.372725 18626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:37533 (error 108)
W20260923 22:11:33.373693 19110 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:11:33.373031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:37533 (local address 127.0.0.1:47590)
0923 22:11:33.373081 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.373091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.373128 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.373648 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37533: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 22:11:33.373838 18626 catalog_manager.cc:4584] TS 42a3d73db92541f49f8f3d911a1d9b29 (127.1.161.129:37533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1aa3503dcb7f40a395fd536333ad2138: Network error: Client connection negotiation failed: client connection to 127.1.161.129:37533: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 22:11:33.373888 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18838
I20260923 22:11:33.378603 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18972
I20260923 22:11:33.383477 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 18610
2026-09-23T22:11:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T22:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:33Z Disabled control of system clock
I20260923 22:11:33.408422 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:37431
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:37431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:33.477733 19145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.477962 19145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.477999 19145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.479094 19145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:33.479156 19145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.479176 19145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.479224 19145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:33.479249 19145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:33.480384 19145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:37431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:33.480618 19145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.480860 19145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.484958 19151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.485054 19145 server_base.cc:1034] running on GCE node
W20260923 22:11:33.484958 19153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.485005 19150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.485396 19145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.485675 19145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.486804 19145 hybrid_clock.cc:648] HybridClock initialized: now 1790201493486791 us; error 29 us; skew 500 ppm
I20260923 22:11:33.487914 19145 webserver.cc:460] Webserver started at http://127.1.161.190:38723/ using document root <none> and password file <none>
I20260923 22:11:33.488139 19145 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.488183 19145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.488307 19145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.489223 19145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "317097ede11d46bbac31116b3fabc369"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "99423228d25719f3283094ca901d860d"
server_key_iv: "0394cd1d85e0e654ff3ef25262c95a1f"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.489509 19145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "317097ede11d46bbac31116b3fabc369"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "99423228d25719f3283094ca901d860d"
server_key_iv: "0394cd1d85e0e654ff3ef25262c95a1f"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.490612 19145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:33.491292 19159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.491490 19145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.491571 19145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "317097ede11d46bbac31116b3fabc369"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "99423228d25719f3283094ca901d860d"
server_key_iv: "0394cd1d85e0e654ff3ef25262c95a1f"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.491639 19145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.509404 19145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.509662 19145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.509783 19145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.512799 19145 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:37431
I20260923 22:11:33.512853 19211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:37431 every 8 connection(s)
I20260923 22:11:33.513185 19145 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:33.514002 19212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.515771 19212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: Bootstrap starting.
I20260923 22:11:33.516194 19212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.516391 19212 log.cc:824] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.516904 19212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: No bootstrap required, opened a new log
I20260923 22:11:33.518282 19212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } }
I20260923 22:11:33.518402 19212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.518424 19212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317097ede11d46bbac31116b3fabc369, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.518548 19212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } }
I20260923 22:11:33.518631 19212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:33.518653 19212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:33.518692 19212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:33.519088 19212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } }
I20260923 22:11:33.519156 19212 leader_election.cc:302] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 317097ede11d46bbac31116b3fabc369; no voters:
I20260923 22:11:33.519315 19212 leader_election.cc:288] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:33.519377 19215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:33.519523 19212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:33.519536 19215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [term 1 LEADER]: Becoming Leader. State: Replica: 317097ede11d46bbac31116b3fabc369, State: Running, Role: LEADER
I20260923 22:11:33.519657 19215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } }
I20260923 22:11:33.520038 19217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 317097ede11d46bbac31116b3fabc369. Latest consensus state: current_term: 1 leader_uuid: "317097ede11d46bbac31116b3fabc369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } } }
I20260923 22:11:33.520129 19217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:33.520467 19224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:33.520897 19224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:33.520273 19216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "317097ede11d46bbac31116b3fabc369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "317097ede11d46bbac31116b3fabc369" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 37431 } } }
I20260923 22:11:33.521448 19216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:33.521935 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19145
I20260923 22:11:33.522027 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:33.522243 1670 external_mini_cluster.cc:1428] Setting key b3681802f87d33d9021abee0ba37ac27
I20260923 22:11:33.522446 19224 catalog_manager.cc:1347] Generated new cluster ID: 2c2ace9303a141d5939cd3c77a5d290c
I20260923 22:11:33.522495 19224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:33.527827 19224 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:33.528313 19224 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:33.532760 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.522758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.1.161.190:37431)
0923 22:11:33.527501 (+ 4743us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.527510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.527558 (+ 48us) server_negotiation.cc:407] Connection header received
0923 22:11:33.527682 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.527689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.527757 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.528529 (+ 772us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.528760 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.529993 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.530981 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.531525 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.531638 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.532193 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.532196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.532196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:33.532362 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.532397 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:33.532408 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.532467 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.532523 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.532584 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4685,"thread_start_us":54,"threads_started":1}
I20260923 22:11:33.536018 19224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: Generated new TSK 0
I20260923 22:11:33.536156 19224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:33.537619 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--builtin_ntp_servers=127.1.161.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:33.609329 19236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.609493 19236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.609510 19236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.610491 19236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.610525 19236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.610549 19236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:33.611778 19236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:33.611974 19236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.612248 19236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.616560 19242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.616698 19244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.616568 19241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.616582 19236 server_base.cc:1034] running on GCE node
I20260923 22:11:33.616978 19236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.617218 19236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.618366 19236 hybrid_clock.cc:648] HybridClock initialized: now 1790201493618341 us; error 39 us; skew 500 ppm
I20260923 22:11:33.619617 19236 webserver.cc:460] Webserver started at http://127.1.161.129:34735/ using document root <none> and password file <none>
I20260923 22:11:33.619846 19236 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.619903 19236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.620025 19236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.621066 19236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "0fa08d2aaabc40b5a1d2242180f59fae"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "761789bd213d4deb0729feb577a017ac"
server_key_iv: "8deccd8d633cd255bbf646db7232c8d6"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.621376 19236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "0fa08d2aaabc40b5a1d2242180f59fae"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "761789bd213d4deb0729feb577a017ac"
server_key_iv: "8deccd8d633cd255bbf646db7232c8d6"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.622504 19236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:33.623153 19250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.623283 19236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:33.623353 19236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "0fa08d2aaabc40b5a1d2242180f59fae"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "761789bd213d4deb0729feb577a017ac"
server_key_iv: "8deccd8d633cd255bbf646db7232c8d6"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.623411 19236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.635887 19236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.636124 19236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.636261 19236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.636452 19236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.636725 19236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.636755 19236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.636773 19236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.636785 19236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.642437 19236 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:40357
I20260923 22:11:33.642480 19363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:40357 every 8 connection(s)
I20260923 22:11:33.642756 19236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:33.647478 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.643399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:52125 (local address 127.1.161.190:37431)
0923 22:11:33.643489 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.643491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.643677 (+ 186us) server_negotiation.cc:407] Connection header received
0923 22:11:33.644125 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.644127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.644138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.644200 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.644831 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.645301 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.646191 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.646469 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.646521 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.647283 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.647284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.647284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:33.647287 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.647298 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:33.647301 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.647339 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.647400 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.647425 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 22:11:33.647663 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.643362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37431 (local address 127.1.161.129:52125)
0923 22:11:33.643556 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.643587 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.644006 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.644275 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.644278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.644286 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.644738 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.644742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.645375 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.645376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.646103 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.646105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.646532 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.646533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.646652 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.647117 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:33.647117 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.647141 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:33.647203 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.647342 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.647344 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.647346 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.647392 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.647463 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260923 22:11:33.648170 19364 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37431
I20260923 22:11:33.648267 19364 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.648432 19364 heartbeater.cc:511] Master 127.1.161.190:37431 requested a full tablet report, sending...
I20260923 22:11:33.648929 19176 ts_manager.cc:194] Registered new tserver with Master: 0fa08d2aaabc40b5a1d2242180f59fae (127.1.161.129:40357)
I20260923 22:11:33.649411 19176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.129:52125
I20260923 22:11:33.651809 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19236
I20260923 22:11:33.651886 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:33.652034 1670 external_mini_cluster.cc:1428] Setting key 5c3da3970b1767c12d03d49f5d8a3d86
I20260923 22:11:33.652745 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--builtin_ntp_servers=127.1.161.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:33.725036 19367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.725196 19367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.725216 19367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.726214 19367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.726249 19367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.726272 19367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:33.727483 19367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:33.727727 19367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.727996 19367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.732229 19373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.732348 19367 server_base.cc:1034] running on GCE node
W20260923 22:11:33.732244 19372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.732234 19375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.732569 19367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.732805 19367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.733958 19367 hybrid_clock.cc:648] HybridClock initialized: now 1790201493733960 us; error 33 us; skew 500 ppm
I20260923 22:11:33.735206 19367 webserver.cc:460] Webserver started at http://127.1.161.130:43225/ using document root <none> and password file <none>
I20260923 22:11:33.735440 19367 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.735504 19367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.735643 19367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.736618 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "8e54e9e50c5045f9942554a1a0b62f21"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "aa99ec0f209c1758ec9b93b1f4a8f9cc"
server_key_iv: "a94021ec58a14ab2b13ca4bed412549b"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.736961 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "8e54e9e50c5045f9942554a1a0b62f21"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "aa99ec0f209c1758ec9b93b1f4a8f9cc"
server_key_iv: "a94021ec58a14ab2b13ca4bed412549b"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.738255 19367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:33.738934 19381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.739086 19367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:33.739221 19367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "8e54e9e50c5045f9942554a1a0b62f21"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "aa99ec0f209c1758ec9b93b1f4a8f9cc"
server_key_iv: "a94021ec58a14ab2b13ca4bed412549b"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.739292 19367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.755128 19367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.755396 19367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.755532 19367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.755728 19367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.756006 19367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.756062 19367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.756100 19367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.756127 19367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.761904 19367 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:41311
I20260923 22:11:33.761950 19494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:41311 every 8 connection(s)
I20260923 22:11:33.762216 19367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:33.766551 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19367
I20260923 22:11:33.766630 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:33.766835 1670 external_mini_cluster.cc:1428] Setting key 80b3c6250ab63d72c6b1b99bde82d3e6
I20260923 22:11:33.766872 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.762791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:47885 (local address 127.1.161.190:37431)
0923 22:11:33.762835 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.762837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.763074 (+ 237us) server_negotiation.cc:407] Connection header received
0923 22:11:33.763618 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.763619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.763632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.763686 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.764344 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.764826 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.765689 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.765856 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.765920 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.766551 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.766552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.766552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:33.766556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.766569 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:33.766573 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.766631 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.766777 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.766805 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 22:11:33.767040 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.762779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37431 (local address 127.1.161.130:47885)
0923 22:11:33.762944 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.762968 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.763503 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.763747 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.763751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.763760 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.764230 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.764234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.764915 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.764916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.765597 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.765599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.765929 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.765930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.766042 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.766429 (+ 387us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:33.766429 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.766451 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:33.766469 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.766655 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.766666 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.766668 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.766776 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.766857 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260923 22:11:33.767719 19495 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37431
I20260923 22:11:33.767791 19495 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.767854 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--builtin_ntp_servers=127.1.161.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:33.767998 19495 heartbeater.cc:511] Master 127.1.161.190:37431 requested a full tablet report, sending...
I20260923 22:11:33.768361 19176 ts_manager.cc:194] Registered new tserver with Master: 8e54e9e50c5045f9942554a1a0b62f21 (127.1.161.130:41311)
I20260923 22:11:33.768872 19176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.130:47885
W20260923 22:11:33.841198 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:33.841380 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:33.841398 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:33.842397 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:33.842430 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:33.842453 19498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:33.843626 19498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:33.843781 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:33.843971 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:33.848163 19504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.848172 19503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:33.848166 19506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:33.848246 19498 server_base.cc:1034] running on GCE node
I20260923 22:11:33.848564 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:33.848819 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:33.849961 19498 hybrid_clock.cc:648] HybridClock initialized: now 1790201493849957 us; error 32 us; skew 500 ppm
I20260923 22:11:33.851019 19498 webserver.cc:460] Webserver started at http://127.1.161.131:43505/ using document root <none> and password file <none>
I20260923 22:11:33.851311 19498 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:33.851385 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:33.851569 19498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:33.852520 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "026d15c2a1ad495eab9e5855e6aace5c"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "2e5195d960f25584bb8211a5376f0936"
server_key_iv: "999f84c1d9977389900ad597d47a53bc"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.852892 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "026d15c2a1ad495eab9e5855e6aace5c"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "2e5195d960f25584bb8211a5376f0936"
server_key_iv: "999f84c1d9977389900ad597d47a53bc"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.854028 19498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:33.854626 19512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.854765 19498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:33.854851 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "026d15c2a1ad495eab9e5855e6aace5c"
format_stamp: "Formatted at 2026-09-23 22:11:33 on dist-test-slave-npvd"
server_key: "2e5195d960f25584bb8211a5376f0936"
server_key_iv: "999f84c1d9977389900ad597d47a53bc"
server_key_version: "encryptionkey@0"
I20260923 22:11:33.854924 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:33.867784 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:33.868026 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:33.868165 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:33.868362 19498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:33.868656 19498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:33.868710 19498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.868748 19498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:33.868784 19498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:33.873797 19498 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:38307
I20260923 22:11:33.873838 19625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:38307 every 8 connection(s)
I20260923 22:11:33.874094 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:33.879026 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.874762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:48237 (local address 127.1.161.190:37431)
0923 22:11:33.874819 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.874821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.875033 (+ 212us) server_negotiation.cc:407] Connection header received
0923 22:11:33.875524 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.875526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.875536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.875603 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.876219 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.876718 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.877762 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.877967 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.878028 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.878819 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.878820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.878820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:33.878824 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.878841 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:33.878845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.878891 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.878943 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.878968 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:33.879207 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.874725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:37431 (local address 127.1.161.131:48237)
0923 22:11:33.874897 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.874932 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.875408 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.875649 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.875652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.875659 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:33.876128 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.876131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.876841 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.876843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.877626 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.877633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.878045 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.878046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.878152 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.878688 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:33.878688 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:33.878722 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:33.878742 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:33.878891 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:33.878893 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:33.878895 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.878938 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.879007 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260923 22:11:33.879666 19626 heartbeater.cc:347] Connected to a master server at 127.1.161.190:37431
I20260923 22:11:33.879765 19626 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:33.879956 19626 heartbeater.cc:511] Master 127.1.161.190:37431 requested a full tablet report, sending...
I20260923 22:11:33.880299 19176 ts_manager.cc:194] Registered new tserver with Master: 026d15c2a1ad495eab9e5855e6aace5c (127.1.161.131:38307)
I20260923 22:11:33.880638 19176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.131:48237
I20260923 22:11:33.882230 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19498
I20260923 22:11:33.882337 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:33.882524 1670 external_mini_cluster.cc:1428] Setting key 047bbff34ad87fae91a83b8f1d45231c
I20260923 22:11:33.883150 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:33.887308 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.884399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.1.161.190:37431)
0923 22:11:33.884433 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.884434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.884556 (+ 122us) server_negotiation.cc:407] Connection header received
0923 22:11:33.884578 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.884578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.884587 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.884626 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:33.884741 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.885341 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.886217 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.886385 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.886463 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.887079 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:33.887080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:33.887080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:33.887084 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:33.887094 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:33.887097 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:33.887135 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.887218 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.887238 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:33.888635 19176 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:33.889999 19176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:33.896754 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.891812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40357 (local address 127.0.0.1:60440)
0923 22:11:33.892391 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.892400 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.892472 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.892678 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.892679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.892680 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.892716 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.892717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.894181 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.894182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.895379 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.895382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.896346 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.896347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.896467 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.896482 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.896595 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.896669 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":61,"threads_started":1}
I20260923 22:11:33.897065 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.892008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.1.161.129:40357)
0923 22:11:33.892162 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.892165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.892448 (+ 283us) server_negotiation.cc:407] Connection header received
0923 22:11:33.892551 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.892553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.892566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.892611 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.892802 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.894098 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.895467 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.896269 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.896344 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.896424 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.896895 (+ 471us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.896966 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":58,"threads_started":1}
I20260923 22:11:33.897816 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.891798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:38307 (local address 127.0.0.1:58612)
0923 22:11:33.892203 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.892226 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.892302 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.892806 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.892807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.892809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.892883 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.892885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.894292 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.894292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.895384 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.895386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.897562 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.897564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.897645 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.897649 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.897711 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.897753 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":84,"thread_start_us":53,"threads_started":1}
I20260923 22:11:33.897981 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.891975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.1.161.131:38307)
0923 22:11:33.892126 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.892130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.892289 (+ 159us) server_negotiation.cc:407] Connection header received
0923 22:11:33.892361 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.892364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.892377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.892498 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.892974 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.894173 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.896456 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.897459 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.897529 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.897603 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.897748 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.897781 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260923 22:11:33.899231 19560 tablet_service.cc:1467] Processing CreateTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 (DEFAULT_TABLE table=test-table [id=b4f86a5f06bb4309a9a22d0e5738cc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.899552 19560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f89a8c75964cafbfa2cb09796b6651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.899572 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.891842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:41311 (local address 127.0.0.1:50892)
0923 22:11:33.895588 (+ 3746us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:33.895597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:33.895645 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:33.896119 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:33.896120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:33.896122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:33.896160 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.896161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.897292 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.897293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.898369 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:33.898372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.899379 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.899380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:33.899434 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.899438 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:33.899483 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:11:33.899515 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3731,"thread_start_us":105,"threads_started":1}
I20260923 22:11:33.899783 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:33.895579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.1.161.130:41311)
0923 22:11:33.895919 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:33.895924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:33.895935 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:33.895970 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:33.895971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:33.895984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:33.896053 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:33.896228 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.897199 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.898461 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:33.899210 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:33.899273 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:33.899347 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:33.899669 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 22:11:33.899711 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":83,"threads_started":1}
I20260923 22:11:33.899686 19298 tablet_service.cc:1467] Processing CreateTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 (DEFAULT_TABLE table=test-table [id=b4f86a5f06bb4309a9a22d0e5738cc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.899977 19298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f89a8c75964cafbfa2cb09796b6651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.902173 19640 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Bootstrap starting.
I20260923 22:11:33.902778 19640 tablet_bootstrap.cc:654] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.902874 19641 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: Bootstrap starting.
I20260923 22:11:33.903072 19640 log.cc:824] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.903009 19429 tablet_service.cc:1467] Processing CreateTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 (DEFAULT_TABLE table=test-table [id=b4f86a5f06bb4309a9a22d0e5738cc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:33.903316 19429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f89a8c75964cafbfa2cb09796b6651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:33.903446 19641 tablet_bootstrap.cc:654] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.903728 19640 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: No bootstrap required, opened a new log
I20260923 22:11:33.903714 19641 log.cc:824] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.903800 19640 ts_tablet_manager.cc:1397] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:33.904376 19641 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: No bootstrap required, opened a new log
I20260923 22:11:33.904438 19641 ts_tablet_manager.cc:1397] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:11:33.905520 19640 raft_consensus.cc:348] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.905674 19640 raft_consensus.cc:374] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.905714 19640 raft_consensus.cc:729] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026d15c2a1ad495eab9e5855e6aace5c, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.905818 19640 consensus_queue.cc:260] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.906039 19640 ts_tablet_manager.cc:1428] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:33.906154 19626 heartbeater.cc:503] Master 127.1.161.190:37431 was elected leader, sending a full tablet report...
I20260923 22:11:33.906246 19645 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Bootstrap starting.
I20260923 22:11:33.906584 19641 raft_consensus.cc:348] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.906735 19641 raft_consensus.cc:374] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.906774 19641 raft_consensus.cc:729] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa08d2aaabc40b5a1d2242180f59fae, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.906795 19645 tablet_bootstrap.cc:654] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:33.906878 19641 consensus_queue.cc:260] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.907070 19645 log.cc:824] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:33.907091 19641 ts_tablet_manager.cc:1428] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 22:11:33.907138 19364 heartbeater.cc:503] Master 127.1.161.190:37431 was elected leader, sending a full tablet report...
I20260923 22:11:33.907752 19645 tablet_bootstrap.cc:492] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: No bootstrap required, opened a new log
I20260923 22:11:33.907830 19645 ts_tablet_manager.cc:1397] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:33.909715 19645 raft_consensus.cc:348] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.909848 19645 raft_consensus.cc:374] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:33.909893 19645 raft_consensus.cc:729] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e54e9e50c5045f9942554a1a0b62f21, State: Initialized, Role: FOLLOWER
I20260923 22:11:33.909999 19645 consensus_queue.cc:260] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:33.910224 19645 ts_tablet_manager.cc:1428] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 22:11:33.910272 19495 heartbeater.cc:503] Master 127.1.161.190:37431 was elected leader, sending a full tablet report...
W20260923 22:11:34.012655 19496 tablet.cc:2369] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:34.107949 19648 raft_consensus.cc:482] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:34.108083 19648 raft_consensus.cc:504] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:34.108443 19648 leader_election.cc:288] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 026d15c2a1ad495eab9e5855e6aace5c (127.1.161.131:38307), 0fa08d2aaabc40b5a1d2242180f59fae (127.1.161.129:40357)
I20260923 22:11:34.111013 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.108623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40357 (local address 127.1.161.130:46691)
0923 22:11:34.108681 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.108693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.108752 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.108940 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.108941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.108943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.109013 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.109015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.109623 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.109624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.110323 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.110325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.110794 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.110861 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.110864 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.110900 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.110927 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":30}
I20260923 22:11:34.111022 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.108702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:46691 (local address 127.1.161.129:40357)
0923 22:11:34.108773 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.108775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.108797 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:11:34.108835 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.108835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.108846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.108912 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.109104 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.109547 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110403 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.110718 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110766 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.110771 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.110921 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.110962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:11:34.111068 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.108706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:49081 (local address 127.1.161.131:38307)
0923 22:11:34.108773 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.108775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.108936 (+ 161us) server_negotiation.cc:407] Connection header received
0923 22:11:34.108973 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.108974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.108983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.109037 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:34.109203 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.109601 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110426 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.110733 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110810 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.110817 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.110971 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.111003 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 22:11:34.111131 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.108625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:38307 (local address 127.1.161.130:49081)
0923 22:11:34.108869 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.108878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.108930 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:34.109046 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:34.109047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:34.109048 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:34.109141 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.109143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.109680 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.109681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.110331 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:34.110332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.110837 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.110838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:34.110902 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.110906 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:34.110955 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:11:34.110982 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":18,"thread_start_us":133,"threads_started":1}
I20260923 22:11:34.111384 19580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f89a8c75964cafbfa2cb09796b6651" candidate_uuid: "8e54e9e50c5045f9942554a1a0b62f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026d15c2a1ad495eab9e5855e6aace5c" is_pre_election: true
I20260923 22:11:34.111367 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f89a8c75964cafbfa2cb09796b6651" candidate_uuid: "8e54e9e50c5045f9942554a1a0b62f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" is_pre_election: true
I20260923 22:11:34.111507 19580 raft_consensus.cc:2454] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e54e9e50c5045f9942554a1a0b62f21 in term 0.
I20260923 22:11:34.111507 19318 raft_consensus.cc:2454] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e54e9e50c5045f9942554a1a0b62f21 in term 0.
I20260923 22:11:34.111730 19382 leader_election.cc:302] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026d15c2a1ad495eab9e5855e6aace5c, 8e54e9e50c5045f9942554a1a0b62f21; no voters:
I20260923 22:11:34.111860 19648 raft_consensus.cc:2781] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:34.111907 19648 raft_consensus.cc:482] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:34.111924 19648 raft_consensus.cc:3037] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.112548 19648 raft_consensus.cc:504] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:34.112679 19648 leader_election.cc:288] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [CANDIDATE]: Term 1 election: Requested vote from peers 026d15c2a1ad495eab9e5855e6aace5c (127.1.161.131:38307), 0fa08d2aaabc40b5a1d2242180f59fae (127.1.161.129:40357)
I20260923 22:11:34.112811 19318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f89a8c75964cafbfa2cb09796b6651" candidate_uuid: "8e54e9e50c5045f9942554a1a0b62f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa08d2aaabc40b5a1d2242180f59fae"
I20260923 22:11:34.112811 19580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f89a8c75964cafbfa2cb09796b6651" candidate_uuid: "8e54e9e50c5045f9942554a1a0b62f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026d15c2a1ad495eab9e5855e6aace5c"
I20260923 22:11:34.112879 19318 raft_consensus.cc:3037] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.112879 19580 raft_consensus.cc:3037] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:34.113673 19318 raft_consensus.cc:2454] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e54e9e50c5045f9942554a1a0b62f21 in term 1.
I20260923 22:11:34.113673 19580 raft_consensus.cc:2454] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e54e9e50c5045f9942554a1a0b62f21 in term 1.
I20260923 22:11:34.113879 19384 leader_election.cc:302] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa08d2aaabc40b5a1d2242180f59fae, 8e54e9e50c5045f9942554a1a0b62f21; no voters:
I20260923 22:11:34.113960 19648 raft_consensus.cc:2781] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:34.114106 19648 raft_consensus.cc:686] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 1 LEADER]: Becoming Leader. State: Replica: 8e54e9e50c5045f9942554a1a0b62f21, State: Running, Role: LEADER
I20260923 22:11:34.114192 19648 consensus_queue.cc:237] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } }
I20260923 22:11:34.114894 19175 catalog_manager.cc:5495] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e54e9e50c5045f9942554a1a0b62f21 (127.1.161.130). New cstate: current_term: 1 leader_uuid: "8e54e9e50c5045f9942554a1a0b62f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e54e9e50c5045f9942554a1a0b62f21" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20260923 22:11:34.124423 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:34.122617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.1.161.130:41311)
0923 22:11:34.122659 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:34.122660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:34.122668 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:34.122718 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:34.122719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:34.122730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:34.122782 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:34.122869 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.123294 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.123907 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:34.124057 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:34.124098 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:34.124210 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:34.124284 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:34.124334 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:34.124360 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 22:11:34.124372 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:11:34.124577 19627 tablet.cc:2369] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:34.125835 19580 raft_consensus.cc:1261] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 1 FOLLOWER]: Refusing update from remote peer 8e54e9e50c5045f9942554a1a0b62f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:34.125846 19318 raft_consensus.cc:1261] T f9f89a8c75964cafbfa2cb09796b6651 P 0fa08d2aaabc40b5a1d2242180f59fae [term 1 FOLLOWER]: Refusing update from remote peer 8e54e9e50c5045f9942554a1a0b62f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:34.126121 19650 consensus_queue.cc:1035] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "026d15c2a1ad495eab9e5855e6aace5c" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:34.126242 19648 consensus_queue.cc:1035] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa08d2aaabc40b5a1d2242180f59fae" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:34.128088 19653 mvcc.cc:205] Tried to move back new op lower bound from 7332665319937466368 to 7332665319892234240. Current Snapshot: MvccSnapshot[applied={T|T < 7332665319937466368}]
I20260923 22:11:34.128803 19655 mvcc.cc:205] Tried to move back new op lower bound from 7332665319937466368 to 7332665319892234240. Current Snapshot: MvccSnapshot[applied={T|T < 7332665319937466368}]
I20260923 22:11:34.128914 19654 mvcc.cc:205] Tried to move back new op lower bound from 7332665319937466368 to 7332665319892234240. Current Snapshot: MvccSnapshot[applied={T|T < 7332665319937466368 or (T in {7332665319937466368})}]
I20260923 22:11:34.130800 19175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:34.130877 19175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:34.131507 19175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 317097ede11d46bbac31116b3fabc369: Sending DeleteTablet for 3 replicas of tablet f9f89a8c75964cafbfa2cb09796b6651
I20260923 22:11:34.131814 19298 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='slave'} at 127.0.0.1:60440
I20260923 22:11:34.131795 19429 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='slave'} at 127.0.0.1:50892
I20260923 22:11:34.131816 19560 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f89a8c75964cafbfa2cb09796b6651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:34 UTC) from {username='slave'} at 127.0.0.1:58612
I20260923 22:11:34.131999 19664 tablet_replica.cc:330] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: stopping tablet replica
I20260923 22:11:34.132055 19665 tablet_replica.cc:330] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: stopping tablet replica
I20260923 22:11:34.132097 19664 raft_consensus.cc:2229] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:34.132146 19665 raft_consensus.cc:2229] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:34.132184 19664 raft_consensus.cc:2258] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:34.132229 19665 raft_consensus.cc:2258] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:34.132277 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19236
I20260923 22:11:34.133018 19664 ts_tablet_manager.cc:1905] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:34.134565 19664 ts_tablet_manager.cc:1918] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:34.134621 19664 log.cc:1196] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/f9f89a8c75964cafbfa2cb09796b6651
I20260923 22:11:34.134847 19664 ts_tablet_manager.cc:1939] T f9f89a8c75964cafbfa2cb09796b6651 P 8e54e9e50c5045f9942554a1a0b62f21: Deleting consensus metadata
I20260923 22:11:34.135249 19161 catalog_manager.cc:4841] TS 8e54e9e50c5045f9942554a1a0b62f21 (127.1.161.130:41311): tablet f9f89a8c75964cafbfa2cb09796b6651 (table test-table [id=b4f86a5f06bb4309a9a22d0e5738cc13]) successfully deleted
W20260923 22:11:34.138280 19162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:40357 (error 108)
I20260923 22:11:34.138592 19665 ts_tablet_manager.cc:1905] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 22:11:34.138729 19635 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:11:34.138572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40357 (local address 127.0.0.1:60456)
0923 22:11:34.138641 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:34.138651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:34.138685 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40357: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":51}
W20260923 22:11:34.138839 19162 catalog_manager.cc:4584] TS 0fa08d2aaabc40b5a1d2242180f59fae (127.1.161.129:40357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9f89a8c75964cafbfa2cb09796b6651: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40357: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260923 22:11:34.139114 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19367
I20260923 22:11:34.140728 19665 ts_tablet_manager.cc:1918] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:34.140821 19665 log.cc:1196] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/f9f89a8c75964cafbfa2cb09796b6651
I20260923 22:11:34.141063 19665 ts_tablet_manager.cc:1939] T f9f89a8c75964cafbfa2cb09796b6651 P 026d15c2a1ad495eab9e5855e6aace5c: Deleting consensus metadata
I20260923 22:11:34.141449 19160 catalog_manager.cc:4841] TS 026d15c2a1ad495eab9e5855e6aace5c (127.1.161.131:38307): tablet f9f89a8c75964cafbfa2cb09796b6651 (table test-table [id=b4f86a5f06bb4309a9a22d0e5738cc13]) successfully deleted
I20260923 22:11:34.143801 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19498
I20260923 22:11:34.147483 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19145
2026-09-23T22:11:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (763 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:34 dist-test-slave-npvd krb5kdc[19669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:34 dist-test-slave-npvd krb5kdc[19669](info): set up 2 sockets
Sep 23 22:11:34 dist-test-slave-npvd krb5kdc[19669](info): commencing operation
krb5kdc: starting...
W20260923 22:11:35.090576 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:35.191097 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36373
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:35.261953 19682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:35.262153 19682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:35.262185 19682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:35.263243 19682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:35.263296 19682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:35.263319 19682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:35.263346 19682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:35.263370 19682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:35.264662 19682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:35.264876 19682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:35.265058 19682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:35.269361 19690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.269367 19687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.269407 19688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:35.269441 19682 server_base.cc:1034] running on GCE node
I20260923 22:11:35.269821 19682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:35.270078 19682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:35.271219 19682 hybrid_clock.cc:648] HybridClock initialized: now 1790201495271212 us; error 34 us; skew 500 ppm
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:35.273504 19682 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:35.273869 19682 webserver.cc:460] Webserver started at http://127.1.161.190:40001/ using document root <none> and password file <none>
I20260923 22:11:35.274091 19682 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:35.274163 19682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:35.274312 19682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:35.275229 19682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "7255589ce957f90e7b7fa895fdd30013"
server_key_iv: "83c71c43c5d33e9984010aa0daee2cd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.275559 19682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "7255589ce957f90e7b7fa895fdd30013"
server_key_iv: "83c71c43c5d33e9984010aa0daee2cd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.276787 19682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:35.277467 19697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.277640 19682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:35.277746 19682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "7255589ce957f90e7b7fa895fdd30013"
server_key_iv: "83c71c43c5d33e9984010aa0daee2cd1"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.277828 19682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:35.289856 19682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:35.291790 19682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:35.291923 19682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:35.295060 19682 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36373
I20260923 22:11:35.295087 19749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36373 every 8 connection(s)
I20260923 22:11:35.295439 19682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:35.296185 19750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.298118 19750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: Bootstrap starting.
I20260923 22:11:35.298576 19750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.298795 19750 log.cc:824] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.299495 19750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: No bootstrap required, opened a new log
I20260923 22:11:35.300891 19750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } }
I20260923 22:11:35.301013 19750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.301035 19750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a78e0f4b2954ebe83f5e9421e05e7fa, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.301170 19750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } }
I20260923 22:11:35.301250 19750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:35.301273 19750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:35.301301 19750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:35.301726 19750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } }
I20260923 22:11:35.301796 19750 leader_election.cc:302] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a78e0f4b2954ebe83f5e9421e05e7fa; no voters:
I20260923 22:11:35.301896 19750 leader_election.cc:288] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:35.301954 19753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:35.302109 19753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [term 1 LEADER]: Becoming Leader. State: Replica: 2a78e0f4b2954ebe83f5e9421e05e7fa, State: Running, Role: LEADER
I20260923 22:11:35.302162 19750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:35.302184 19753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } }
I20260923 22:11:35.302482 19755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a78e0f4b2954ebe83f5e9421e05e7fa. Latest consensus state: current_term: 1 leader_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } } }
I20260923 22:11:35.302562 19755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:35.302585 19754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a78e0f4b2954ebe83f5e9421e05e7fa" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36373 } } }
I20260923 22:11:35.302627 19754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:35.303144 19760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:35.303573 19760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:35.304453 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19682
I20260923 22:11:35.304562 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:35.304735 1670 external_mini_cluster.cc:1428] Setting key 587f72b6c37dd324515582bfd7f92a39
I20260923 22:11:35.304924 19760 catalog_manager.cc:1347] Generated new cluster ID: 083efb0643e3490e8fefe3820328156a
I20260923 22:11:35.304978 19760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:35.314594 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.305236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.1.161.190:36373)
0923 22:11:35.305347 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.305355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.305380 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:11:35.305468 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.305473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.305523 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.305638 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.306308 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.307537 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.308605 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.309171 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.309251 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.311593 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.311597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.311597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:35.311607 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.313231 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.313560 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.313563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.313563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.313660 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.313831 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.313831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.313832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.314074 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.314210 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.314277 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.314387 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":54,"threads_started":1}
I20260923 22:11:35.326119 19760 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:35.326558 19760 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:35.332901 19760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: Generated new TSK 0
I20260923 22:11:35.333052 19760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:11:35.357077 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:35.429199 19776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:35.429365 19776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:35.429385 19776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:35.430362 19776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:35.430393 19776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:35.430415 19776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:35.431663 19776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:35.431813 19776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:35.431951 19776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:35.436286 19782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.436273 19781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:35.436300 19776 server_base.cc:1034] running on GCE node
W20260923 22:11:35.436265 19784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:35.436674 19776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:35.436928 19776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:35.438062 19776 hybrid_clock.cc:648] HybridClock initialized: now 1790201495438044 us; error 35 us; skew 500 ppm
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:35.440402 19776 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:11:35.440809 19776 webserver.cc:460] Webserver started at http://127.1.161.129:41271/ using document root <none> and password file <none>
I20260923 22:11:35.441004 19776 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:35.441063 19776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:35.441174 19776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:35.442025 19776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "2a0969344acb4083954f6de701369193"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "54399be8adbfd8358bc9b5a28b2af77e"
server_key_iv: "aa3a3ccbd3f4ee5e53cfe9d2b7892553"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.442343 19776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "2a0969344acb4083954f6de701369193"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "54399be8adbfd8358bc9b5a28b2af77e"
server_key_iv: "aa3a3ccbd3f4ee5e53cfe9d2b7892553"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.443536 19776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:11:35.444119 19791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.444247 19776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:35.444353 19776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "2a0969344acb4083954f6de701369193"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "54399be8adbfd8358bc9b5a28b2af77e"
server_key_iv: "aa3a3ccbd3f4ee5e53cfe9d2b7892553"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.444424 19776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:35.465732 19776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:35.467633 19776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:35.467792 19776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:35.467988 19776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:35.468269 19776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:35.468322 19776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.468355 19776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:35.468382 19776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.473364 19776 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:45835
I20260923 22:11:35.473426 19904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:45835 every 8 connection(s)
I20260923 22:11:35.473683 19776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:35.481268 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19776
I20260923 22:11:35.481370 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:35.481585 1670 external_mini_cluster.cc:1428] Setting key 7e13b1c28795f21fa1e39f88a100dd54
I20260923 22:11:35.482095 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.474280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:42191 (local address 127.1.161.190:36373)
0923 22:11:35.474319 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.474321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.474489 (+ 168us) server_negotiation.cc:407] Connection header received
0923 22:11:35.474915 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.474917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.474931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.474983 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.475804 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.476264 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.477188 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.477391 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.477460 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.479495 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.479496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.479497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:35.479504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.480727 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.481067 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.481068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.481068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.481143 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.481466 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.481467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.481468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.481593 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.481657 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.481939 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.482018 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:35.482231 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.474245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36373 (local address 127.1.161.129:42191)
0923 22:11:35.474372 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.474417 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.474820 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.475040 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.475043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.475551 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:35.475707 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.475712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.476352 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.476353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.477095 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.477099 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.477467 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.477468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.477588 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.478087 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:35.478095 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:35.479406 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:35.480801 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.480807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.480809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.480994 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.481235 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.481236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.481236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.481360 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.481686 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:35.481692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:35.481813 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.481905 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.482042 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260923 22:11:35.482640 19905 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36373
I20260923 22:11:35.482735 19905 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:35.482882 19905 heartbeater.cc:511] Master 127.1.161.190:36373 requested a full tablet report, sending...
I20260923 22:11:35.483420 19714 ts_manager.cc:194] Registered new tserver with Master: 2a0969344acb4083954f6de701369193 (127.1.161.129:45835)
I20260923 22:11:35.484215 19714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:42191
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:11:35.503350 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:35.574513 19910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:35.574692 19910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:35.574714 19910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:35.575845 19910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:35.575883 19910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:35.575907 19910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:35.577302 19910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:35.577530 19910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:35.577759 19910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:35.582430 19916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.582444 19918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.582444 19915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:35.582681 19910 server_base.cc:1034] running on GCE node
I20260923 22:11:35.582850 19910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:35.583127 19910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:35.584282 19910 hybrid_clock.cc:648] HybridClock initialized: now 1790201495584273 us; error 28 us; skew 500 ppm
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:35.586295 19910 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:11:35.586714 19910 webserver.cc:460] Webserver started at http://127.1.161.130:35373/ using document root <none> and password file <none>
I20260923 22:11:35.586916 19910 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:35.586974 19910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:35.587098 19910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:35.588053 19910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "0f46e1f26d1d47cc8364f55845060e35"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "6896c80ad62c960469df73e3a2300dfb"
server_key_iv: "b18b6ceb70c10beb91dc5cf3811c328a"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.588379 19910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "0f46e1f26d1d47cc8364f55845060e35"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "6896c80ad62c960469df73e3a2300dfb"
server_key_iv: "b18b6ceb70c10beb91dc5cf3811c328a"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.589583 19910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:35.590166 19925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.590325 19910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:35.590408 19910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "0f46e1f26d1d47cc8364f55845060e35"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "6896c80ad62c960469df73e3a2300dfb"
server_key_iv: "b18b6ceb70c10beb91dc5cf3811c328a"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.590483 19910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:35.612136 19910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:35.613740 19910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:35.613904 19910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:35.614110 19910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:35.614379 19910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:35.614430 19910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.614468 19910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:35.614495 19910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.619895 19910 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:41731
I20260923 22:11:35.619928 20038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:41731 every 8 connection(s)
I20260923 22:11:35.620203 19910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:35.627311 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 19910
I20260923 22:11:35.627430 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:35.627627 1670 external_mini_cluster.cc:1428] Setting key 42bce220fc06bc2e43f559c9881a27d1
I20260923 22:11:35.629324 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.620838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:56489 (local address 127.1.161.190:36373)
0923 22:11:35.620909 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.620911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.621099 (+ 188us) server_negotiation.cc:407] Connection header received
0923 22:11:35.621577 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.621578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.621590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.621643 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.622526 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.623012 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.623980 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.624191 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.624259 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.626360 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.626361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.626362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:35.626369 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.627812 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.628341 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.628341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.628342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.628408 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.628692 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.628693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.628693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.628794 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.628844 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.629199 (+ 355us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.629268 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:35.629592 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.620806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36373 (local address 127.1.161.130:56489)
0923 22:11:35.620967 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.620996 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.621457 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.621718 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.621721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.622247 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:35.622427 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.622431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.623111 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.623112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.623893 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.623895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.624263 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.624264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.624385 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.624826 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:35.624834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:35.626256 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:35.627889 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.627896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.627898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.628197 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.628483 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.628484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.628484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.628623 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.628872 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:35.628878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:35.629049 (+ 171us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.629183 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.629341 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260923 22:11:35.630050 20039 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36373
I20260923 22:11:35.630127 20039 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:35.630267 20039 heartbeater.cc:511] Master 127.1.161.190:36373 requested a full tablet report, sending...
I20260923 22:11:35.630602 19714 ts_manager.cc:194] Registered new tserver with Master: 0f46e1f26d1d47cc8364f55845060e35 (127.1.161.130:41731)
I20260923 22:11:35.630932 19714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:56489
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:11:35.650395 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:35.724766 20044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:35.724951 20044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:35.724983 20044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:35.725935 20044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:35.725996 20044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:35.726032 20044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:35.727304 20044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:35.727514 20044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:35.727726 20044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:35.732331 20052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.732350 20049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:35.732337 20050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:35.732350 20044 server_base.cc:1034] running on GCE node
I20260923 22:11:35.732748 20044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:35.732995 20044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:35.734170 20044 hybrid_clock.cc:648] HybridClock initialized: now 1790201495734160 us; error 23 us; skew 500 ppm
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:35.736354 20044 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:11:35.736711 20044 webserver.cc:460] Webserver started at http://127.1.161.131:41757/ using document root <none> and password file <none>
I20260923 22:11:35.736963 20044 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:35.737026 20044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:35.737169 20044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:35.738106 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "22282f134e4d493abdf41895294befa1"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "def9a668dbebc08990679dc5b37b5ef6"
server_key_iv: "4bb5598cf9588a439e35b04db59fa01e"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.738436 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "22282f134e4d493abdf41895294befa1"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "def9a668dbebc08990679dc5b37b5ef6"
server_key_iv: "4bb5598cf9588a439e35b04db59fa01e"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.739674 20044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 22:11:35.740266 20059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.740402 20044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:35.740492 20044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "22282f134e4d493abdf41895294befa1"
format_stamp: "Formatted at 2026-09-23 22:11:35 on dist-test-slave-npvd"
server_key: "def9a668dbebc08990679dc5b37b5ef6"
server_key_iv: "4bb5598cf9588a439e35b04db59fa01e"
server_key_version: "encryptionkey@0"
I20260923 22:11:35.740552 20044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:35.759624 20044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:35.761688 20044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:35.761847 20044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:35.762048 20044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:35.762315 20044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:35.762375 20044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.762411 20044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:35.762436 20044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:35.767373 20044 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:33637
I20260923 22:11:35.767417 20172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:33637 every 8 connection(s)
I20260923 22:11:35.767666 20044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:11:35 dist-test-slave-npvd krb5kdc[19669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201495, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:35.774394 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20044
I20260923 22:11:35.774528 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:35.774719 1670 external_mini_cluster.cc:1428] Setting key f4d38c42f1c1eaa3ba4db7ef995174dc
I20260923 22:11:35.775534 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.768363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:45029 (local address 127.1.161.190:36373)
0923 22:11:35.768439 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.768441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.768597 (+ 156us) server_negotiation.cc:407] Connection header received
0923 22:11:35.769004 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.769005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.769015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.769060 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.769762 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.770192 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.770995 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.771163 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.771247 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.772903 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.772904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.772904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:35.772909 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.773918 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.774367 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.774368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.774368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.774414 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.774707 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.774708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.774708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.774940 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.775018 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.775378 (+ 360us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.775449 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 22:11:35.775775 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.768315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36373 (local address 127.1.161.131:45029)
0923 22:11:35.768482 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.768512 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.768920 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.769103 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.769105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.769542 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:35.769697 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.769701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.770260 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.770260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.770929 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.770931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.771258 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.771259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.771338 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.771671 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:35.771677 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:35.772830 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:35.774012 (+ 1182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.774017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.774019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.774287 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.774478 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:35.774479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:35.774480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:35.774627 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:35.774998 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:35.775005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:35.775169 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.775320 (+ 151us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.775460 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260923 22:11:35.776144 20173 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36373
I20260923 22:11:35.776217 20173 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:35.776328 20173 heartbeater.cc:511] Master 127.1.161.190:36373 requested a full tablet report, sending...
I20260923 22:11:35.776625 19714 ts_manager.cc:194] Registered new tserver with Master: 22282f134e4d493abdf41895294befa1 (127.1.161.131:33637)
I20260923 22:11:35.776973 19714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:45029
I20260923 22:11:35.786334 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:35.793098 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.787598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.1.161.190:36373)
0923 22:11:35.787636 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.787637 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.787745 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:11:35.787779 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.787780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.787787 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.787828 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:35.788356 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.788796 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.789524 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.789691 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.789755 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.790775 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:35.790776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:35.790776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:35.790781 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:35.792159 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.792430 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.792431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.792431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.792515 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:35.792723 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:35.792723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:35.792724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:35.792827 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:35.792900 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.792987 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.793048 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:11:35.794461 19714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:35.795885 19714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:35.801576 20184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.797652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33637 (local address 127.0.0.1:44206)
0923 22:11:35.798077 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.798114 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.798232 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.798417 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.798419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.798422 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.798480 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.798482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799748 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.799749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.800700 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.800703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801356 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.801402 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801409 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.801472 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.801518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260923 22:11:35.801585 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.797706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.1.161.131:33637)
0923 22:11:35.797883 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.797886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.798216 (+ 330us) server_negotiation.cc:407] Connection header received
0923 22:11:35.798307 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.798309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.798323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.798415 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.798646 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799662 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.800780 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801225 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801255 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801302 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.801495 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.801526 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260923 22:11:35.801831 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.797901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:41731 (local address 127.0.0.1:46056)
0923 22:11:35.798444 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.798452 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.798499 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.798741 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.798742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.798744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.798784 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.798786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799976 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.799977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.800699 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.800701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801576 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.801630 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801633 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.801682 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.801751 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":77,"threads_started":1}
I20260923 22:11:35.802008 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.797695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:45835 (local address 127.0.0.1:37936)
0923 22:11:35.798264 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:35.798274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:35.798356 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:35.798601 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:35.798602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:35.798605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:35.798654 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.798656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799601 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.799602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.800840 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:35.800841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801595 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:35.801763 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801766 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:35.801805 (+ 39us) client_negotiation.cc:240] Negotiation successful
0923 22:11:35.801833 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":25,"thread_start_us":92,"threads_started":1}
I20260923 22:11:35.802191 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.797914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.1.161.129:45835)
0923 22:11:35.798082 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.798086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.798349 (+ 263us) server_negotiation.cc:407] Connection header received
0923 22:11:35.798427 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.798429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.798446 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.798525 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.798731 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799510 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.800892 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801472 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801531 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801572 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.801870 (+ 298us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.801991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260923 22:11:35.802260 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:35.798108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.1.161.130:41731)
0923 22:11:35.798257 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:35.798261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:35.798515 (+ 254us) server_negotiation.cc:407] Connection header received
0923 22:11:35.798565 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:35.798568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:35.798583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:35.798657 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:35.798865 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.799893 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.800780 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:35.801432 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:35.801497 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:35.801545 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:35.801986 (+ 441us) server_negotiation.cc:299] Negotiation successful
0923 22:11:35.802168 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260923 22:11:35.802282 20107 tablet_service.cc:1467] Processing CreateTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 (DEFAULT_TABLE table=test-table [id=d2f254c723dc458894359dbb56072f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.802595 20107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a383c43cfea471e87c9432da1a7a5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.802816 19973 tablet_service.cc:1467] Processing CreateTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 (DEFAULT_TABLE table=test-table [id=d2f254c723dc458894359dbb56072f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.802845 19839 tablet_service.cc:1467] Processing CreateTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 (DEFAULT_TABLE table=test-table [id=d2f254c723dc458894359dbb56072f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:35.803150 19839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a383c43cfea471e87c9432da1a7a5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.803144 19973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a383c43cfea471e87c9432da1a7a5b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:35.804903 20187 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Bootstrap starting.
I20260923 22:11:35.805426 20187 tablet_bootstrap.cc:654] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.805787 20187 log.cc:824] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.806008 20188 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Bootstrap starting.
I20260923 22:11:35.806283 20189 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Bootstrap starting.
I20260923 22:11:35.806598 20188 tablet_bootstrap.cc:654] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.806782 20189 tablet_bootstrap.cc:654] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:35.807035 20188 log.cc:824] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.807247 20189 log.cc:824] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:35.807667 20188 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: No bootstrap required, opened a new log
I20260923 22:11:35.807739 20188 ts_tablet_manager.cc:1397] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:11:35.807920 20189 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: No bootstrap required, opened a new log
I20260923 22:11:35.807993 20189 ts_tablet_manager.cc:1397] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 22:11:35.809309 20188 raft_consensus.cc:348] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.809420 20188 raft_consensus.cc:374] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.809440 20188 raft_consensus.cc:729] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f46e1f26d1d47cc8364f55845060e35, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.809407 20189 raft_consensus.cc:348] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.809573 20189 raft_consensus.cc:374] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.809551 20188 consensus_queue.cc:260] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.809610 20189 raft_consensus.cc:729] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a0969344acb4083954f6de701369193, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.809705 20189 consensus_queue.cc:260] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.809785 20188 ts_tablet_manager.cc:1428] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:35.809813 20039 heartbeater.cc:503] Master 127.1.161.190:36373 was elected leader, sending a full tablet report...
I20260923 22:11:35.810202 20189 ts_tablet_manager.cc:1428] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:35.810254 19905 heartbeater.cc:503] Master 127.1.161.190:36373 was elected leader, sending a full tablet report...
I20260923 22:11:35.810446 20187 tablet_bootstrap.cc:492] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: No bootstrap required, opened a new log
I20260923 22:11:35.810521 20187 ts_tablet_manager.cc:1397] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 22:11:35.811836 20187 raft_consensus.cc:348] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.811949 20187 raft_consensus.cc:374] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:35.811967 20187 raft_consensus.cc:729] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22282f134e4d493abdf41895294befa1, State: Initialized, Role: FOLLOWER
I20260923 22:11:35.812060 20187 consensus_queue.cc:260] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:35.812247 20187 ts_tablet_manager.cc:1428] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:35.812296 20173 heartbeater.cc:503] Master 127.1.161.190:36373 was elected leader, sending a full tablet report...
W20260923 22:11:35.870672 20040 tablet.cc:2369] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:35.974340 19906 tablet.cc:2369] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 22:11:36.018226 20174 tablet.cc:2369] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:36.137606 20194 raft_consensus.cc:482] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:36.137750 20194 raft_consensus.cc:504] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:36.138119 20194 leader_election.cc:288] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22282f134e4d493abdf41895294befa1 (127.1.161.131:33637), 0f46e1f26d1d47cc8364f55845060e35 (127.1.161.130:41731)
I20260923 22:11:36.141014 20197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.138502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:41731 (local address 127.1.161.129:43747)
0923 22:11:36.138600 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.138610 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.138681 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.138803 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.138804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.138806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.138902 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.138904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.139541 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.139542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.140288 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.140290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.140773 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.140837 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.140841 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.140887 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.140915 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":39,"threads_started":1}
I20260923 22:11:36.141026 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.138570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:43747 (local address 127.1.161.130:41731)
0923 22:11:36.138619 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.138621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.138672 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:11:36.138732 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.138733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.138741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.138793 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.139009 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.139461 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140373 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.140702 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140753 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.140760 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.140909 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.140954 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:11:36.141068 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.138419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:50171 (local address 127.1.161.131:33637)
0923 22:11:36.138483 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.138499 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.138511 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:11:36.138614 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.138616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.138628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.138660 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.138904 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.139461 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140377 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.140724 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140806 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.140813 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.140965 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.141000 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:11:36.141119 20196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.138323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:33637 (local address 127.1.161.129:50171)
0923 22:11:36.138434 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.138446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.138513 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.138737 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.138738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.138741 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.138821 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.138824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.139541 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.139542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.140289 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.140291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.140824 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.140824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.140889 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.140893 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.140946 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.140980 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":39,"thread_start_us":66,"threads_started":1}
I20260923 22:11:36.141425 19993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a383c43cfea471e87c9432da1a7a5b3" candidate_uuid: "2a0969344acb4083954f6de701369193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f46e1f26d1d47cc8364f55845060e35" is_pre_election: true
I20260923 22:11:36.141552 19993 raft_consensus.cc:2454] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a0969344acb4083954f6de701369193 in term 0.
I20260923 22:11:36.141768 19792 leader_election.cc:302] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f46e1f26d1d47cc8364f55845060e35, 2a0969344acb4083954f6de701369193; no voters:
I20260923 22:11:36.141848 20127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a383c43cfea471e87c9432da1a7a5b3" candidate_uuid: "2a0969344acb4083954f6de701369193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22282f134e4d493abdf41895294befa1" is_pre_election: true
I20260923 22:11:36.141876 20194 raft_consensus.cc:2781] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:36.141932 20194 raft_consensus.cc:482] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:36.141953 20194 raft_consensus.cc:3037] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.141981 20127 raft_consensus.cc:2454] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a0969344acb4083954f6de701369193 in term 0.
I20260923 22:11:36.142616 20194 raft_consensus.cc:504] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:36.142732 20194 leader_election.cc:288] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [CANDIDATE]: Term 1 election: Requested vote from peers 22282f134e4d493abdf41895294befa1 (127.1.161.131:33637), 0f46e1f26d1d47cc8364f55845060e35 (127.1.161.130:41731)
I20260923 22:11:36.142889 19993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a383c43cfea471e87c9432da1a7a5b3" candidate_uuid: "2a0969344acb4083954f6de701369193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f46e1f26d1d47cc8364f55845060e35"
I20260923 22:11:36.142889 20127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a383c43cfea471e87c9432da1a7a5b3" candidate_uuid: "2a0969344acb4083954f6de701369193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22282f134e4d493abdf41895294befa1"
I20260923 22:11:36.142959 20127 raft_consensus.cc:3037] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.142959 19993 raft_consensus.cc:3037] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.143851 20127 raft_consensus.cc:2454] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a0969344acb4083954f6de701369193 in term 1.
I20260923 22:11:36.143851 19993 raft_consensus.cc:2454] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a0969344acb4083954f6de701369193 in term 1.
I20260923 22:11:36.144003 19792 leader_election.cc:302] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f46e1f26d1d47cc8364f55845060e35, 2a0969344acb4083954f6de701369193; no voters:
I20260923 22:11:36.144078 20194 raft_consensus.cc:2781] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:36.144208 20194 raft_consensus.cc:686] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 1 LEADER]: Becoming Leader. State: Replica: 2a0969344acb4083954f6de701369193, State: Running, Role: LEADER
I20260923 22:11:36.144301 20194 consensus_queue.cc:237] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } }
I20260923 22:11:36.145052 19714 catalog_manager.cc:5495] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a0969344acb4083954f6de701369193 (127.1.161.129). New cstate: current_term: 1 leader_uuid: "2a0969344acb4083954f6de701369193" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a0969344acb4083954f6de701369193" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 45835 } health_report { overall_health: HEALTHY } } }
I20260923 22:11:36.154994 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.153076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.1.161.129:45835)
0923 22:11:36.153129 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.153131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.153138 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:11:36.153163 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.153164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.153179 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.153241 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:36.153345 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.153726 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.154386 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.154565 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.154613 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.154754 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:36.154830 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:36.154901 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.154944 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.154952 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:11:36.156452 20127 raft_consensus.cc:1261] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 1 FOLLOWER]: Refusing update from remote peer 2a0969344acb4083954f6de701369193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:36.156479 19993 raft_consensus.cc:1261] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 1 FOLLOWER]: Refusing update from remote peer 2a0969344acb4083954f6de701369193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:36.156795 20194 consensus_queue.cc:1035] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f46e1f26d1d47cc8364f55845060e35" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 41731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:36.156950 20198 consensus_queue.cc:1035] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22282f134e4d493abdf41895294befa1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 33637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:36.159390 20203 mvcc.cc:205] Tried to move back new op lower bound from 7332665328254607360 to 7332665328207536128. Current Snapshot: MvccSnapshot[applied={T|T < 7332665328254607360}]
I20260923 22:11:36.159513 20201 mvcc.cc:205] Tried to move back new op lower bound from 7332665328254607360 to 7332665328207536128. Current Snapshot: MvccSnapshot[applied={T|T < 7332665328254607360}]
I20260923 22:11:36.159673 20202 mvcc.cc:205] Tried to move back new op lower bound from 7332665328254607360 to 7332665328207536128. Current Snapshot: MvccSnapshot[applied={T|T < 7332665328254607360 or (T in {7332665328254607360})}]
I20260923 22:11:36.161198 19713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:36.161281 19713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:36.161921 19713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a78e0f4b2954ebe83f5e9421e05e7fa: Sending DeleteTablet for 3 replicas of tablet 7a383c43cfea471e87c9432da1a7a5b3
I20260923 22:11:36.162272 19973 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:46056
I20260923 22:11:36.162338 20107 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:44206
I20260923 22:11:36.162410 19839 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a383c43cfea471e87c9432da1a7a5b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:37936
I20260923 22:11:36.162560 20213 tablet_replica.cc:330] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193: stopping tablet replica
I20260923 22:11:36.162563 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19776
I20260923 22:11:36.162642 20213 raft_consensus.cc:2229] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:36.162760 20213 raft_consensus.cc:2258] T 7a383c43cfea471e87c9432da1a7a5b3 P 2a0969344acb4083954f6de701369193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:36.162884 20211 tablet_replica.cc:330] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: stopping tablet replica
I20260923 22:11:36.162999 20211 raft_consensus.cc:2229] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:36.163045 20212 tablet_replica.cc:330] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: stopping tablet replica
I20260923 22:11:36.163092 20211 raft_consensus.cc:2258] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:36.163115 20212 raft_consensus.cc:2229] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:36.163185 20212 raft_consensus.cc:2258] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:36.163569 20211 ts_tablet_manager.cc:1905] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:36.163596 20212 ts_tablet_manager.cc:1905] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:36.165131 20211 ts_tablet_manager.cc:1918] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:36.165148 20212 ts_tablet_manager.cc:1918] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:36.165198 20211 log.cc:1196] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/7a383c43cfea471e87c9432da1a7a5b3
I20260923 22:11:36.165198 20212 log.cc:1196] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/7a383c43cfea471e87c9432da1a7a5b3
I20260923 22:11:36.165427 20211 ts_tablet_manager.cc:1939] T 7a383c43cfea471e87c9432da1a7a5b3 P 0f46e1f26d1d47cc8364f55845060e35: Deleting consensus metadata
I20260923 22:11:36.165427 20212 ts_tablet_manager.cc:1939] T 7a383c43cfea471e87c9432da1a7a5b3 P 22282f134e4d493abdf41895294befa1: Deleting consensus metadata
I20260923 22:11:36.165758 19698 catalog_manager.cc:4841] TS 0f46e1f26d1d47cc8364f55845060e35 (127.1.161.130:41731): tablet 7a383c43cfea471e87c9432da1a7a5b3 (table test-table [id=d2f254c723dc458894359dbb56072f0d]) successfully deleted
I20260923 22:11:36.165838 19698 catalog_manager.cc:4841] TS 22282f134e4d493abdf41895294befa1 (127.1.161.131:33637): tablet 7a383c43cfea471e87c9432da1a7a5b3 (table test-table [id=d2f254c723dc458894359dbb56072f0d]) successfully deleted
W20260923 22:11:36.168877 19699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:45835 (error 108)
I20260923 22:11:36.169126 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19910
I20260923 22:11:36.169286 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.169182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:45835 (local address 127.0.0.1:37948)
0923 22:11:36.169226 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.169256 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:45835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:11:36.169373 19699 catalog_manager.cc:4584] TS 2a0969344acb4083954f6de701369193 (127.1.161.129:45835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a383c43cfea471e87c9432da1a7a5b3: Network error: Client connection negotiation failed: client connection to 127.1.161.129:45835: connect: Connection refused (error 111)
I20260923 22:11:36.173970 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20044
I20260923 22:11:36.178228 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 19682
2026-09-23T22:11:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2030 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T22:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:36Z Disabled control of system clock
I20260923 22:11:36.202831 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40679
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:40679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:36.272789 20218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.272966 20218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.272984 20218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.273924 20218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:36.273952 20218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.273963 20218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.273973 20218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:36.273983 20218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:36.275110 20218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:40679
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:40679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:36.275343 20218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.275645 20218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.280022 20223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.280059 20224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.280063 20226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.280100 20218 server_base.cc:1034] running on GCE node
I20260923 22:11:36.280508 20218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.280799 20218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.281955 20218 hybrid_clock.cc:648] HybridClock initialized: now 1790201496281936 us; error 32 us; skew 500 ppm
I20260923 22:11:36.282941 20218 webserver.cc:460] Webserver started at http://127.1.161.190:41201/ using document root <none> and password file <none>
I20260923 22:11:36.283149 20218 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.283241 20218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.283363 20218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.284247 20218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "40fad4ecbeab4ea39debc11882e6ad54"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "61e44a29282960073f5b3383d050bf88"
server_key_iv: "51284a44d259e2f0f215ac550532ef7c"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.284565 20218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "40fad4ecbeab4ea39debc11882e6ad54"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "61e44a29282960073f5b3383d050bf88"
server_key_iv: "51284a44d259e2f0f215ac550532ef7c"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.285614 20218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:36.286208 20232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.286350 20218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:36.286432 20218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "40fad4ecbeab4ea39debc11882e6ad54"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "61e44a29282960073f5b3383d050bf88"
server_key_iv: "51284a44d259e2f0f215ac550532ef7c"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.286507 20218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.301972 20218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.302217 20218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.302353 20218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.305941 20218 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:40679
I20260923 22:11:36.305968 20284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:40679 every 8 connection(s)
I20260923 22:11:36.306253 20218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:36.306563 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20218
I20260923 22:11:36.306685 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:36.306876 1670 external_mini_cluster.cc:1428] Setting key 4bce600302034a2d157119a9fa7a95a2
I20260923 22:11:36.307269 20285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:36.309717 20285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: Bootstrap starting.
I20260923 22:11:36.310225 20285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:36.310498 20285 log.cc:824] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:36.311077 20285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: No bootstrap required, opened a new log
I20260923 22:11:36.311985 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.307332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.1.161.190:40679)
0923 22:11:36.307552 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.307560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.307591 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:11:36.308216 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.308223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.308278 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.308419 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.308631 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.309448 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.310228 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.310723 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.310838 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.311447 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.311451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.311451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:36.311598 (+ 147us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.311625 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:36.311633 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.311714 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.311778 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.311843 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260923 22:11:36.312937 20285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } }
I20260923 22:11:36.313066 20285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:36.313113 20285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fad4ecbeab4ea39debc11882e6ad54, State: Initialized, Role: FOLLOWER
I20260923 22:11:36.313216 20285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } }
I20260923 22:11:36.313298 20285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:36.313335 20285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:36.313375 20285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.313802 20285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } }
I20260923 22:11:36.313902 20285 leader_election.cc:302] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40fad4ecbeab4ea39debc11882e6ad54; no voters:
I20260923 22:11:36.314064 20285 leader_election.cc:288] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:36.314109 20290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:36.314245 20290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [term 1 LEADER]: Becoming Leader. State: Replica: 40fad4ecbeab4ea39debc11882e6ad54, State: Running, Role: LEADER
I20260923 22:11:36.314327 20290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } }
I20260923 22:11:36.314352 20285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:36.314627 20290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40fad4ecbeab4ea39debc11882e6ad54. Latest consensus state: current_term: 1 leader_uuid: "40fad4ecbeab4ea39debc11882e6ad54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } } }
I20260923 22:11:36.314710 20290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:36.314632 20292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40fad4ecbeab4ea39debc11882e6ad54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40fad4ecbeab4ea39debc11882e6ad54" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 40679 } } }
I20260923 22:11:36.314870 20292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:36.315784 20294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:36.316253 20294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:36.317538 20294 catalog_manager.cc:1347] Generated new cluster ID: e8536d10de8b4a75b7aa23f22b764927
I20260923 22:11:36.317606 20294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:36.326094 20294 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:36.326519 20294 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:36.334515 20294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: Generated new TSK 0
I20260923 22:11:36.334687 20294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:36.337585 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--builtin_ntp_servers=127.1.161.148:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:36.419320 20309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.419486 20309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.419505 20309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.420607 20309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.420643 20309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.420668 20309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:36.421918 20309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:36.422122 20309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.422398 20309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.427505 20315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.427557 20309 server_base.cc:1034] running on GCE node
W20260923 22:11:36.427664 20314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.427501 20317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.428033 20309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.428284 20309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.429448 20309 hybrid_clock.cc:648] HybridClock initialized: now 1790201496429435 us; error 31 us; skew 500 ppm
I20260923 22:11:36.430660 20309 webserver.cc:460] Webserver started at http://127.1.161.129:44553/ using document root <none> and password file <none>
I20260923 22:11:36.430881 20309 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.430944 20309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.431079 20309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.432039 20309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "5758fe0ce5954b39a96adeeb407e4f70"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "4ed8361a161a2cf53b6e586de1e186d4"
server_key_iv: "1e7c6c157c856d620a03ef64fbdfbdba"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.432380 20309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "5758fe0ce5954b39a96adeeb407e4f70"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "4ed8361a161a2cf53b6e586de1e186d4"
server_key_iv: "1e7c6c157c856d620a03ef64fbdfbdba"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.433614 20309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:36.434329 20323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.434525 20309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:36.434616 20309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "5758fe0ce5954b39a96adeeb407e4f70"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "4ed8361a161a2cf53b6e586de1e186d4"
server_key_iv: "1e7c6c157c856d620a03ef64fbdfbdba"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.434677 20309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.449095 20309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.449275 20309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.449357 20309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.449494 20309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.449703 20309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.449729 20309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.449747 20309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.449757 20309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.454805 20309 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:46565
I20260923 22:11:36.454890 20436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:46565 every 8 connection(s)
I20260923 22:11:36.455135 20309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:36.459661 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.455744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:44447 (local address 127.1.161.190:40679)
0923 22:11:36.455795 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.455797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.455987 (+ 190us) server_negotiation.cc:407] Connection header received
0923 22:11:36.456411 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.456412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.456425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.456471 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.457022 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.457447 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.458364 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.458574 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.458649 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.459404 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.459405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.459406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:36.459410 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.459423 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:36.459427 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.459474 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.459535 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.459567 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:11:36.459829 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.455744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40679 (local address 127.1.161.129:44447)
0923 22:11:36.455894 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.455920 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.456320 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.456525 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.456528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.456536 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:36.456944 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.456947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.457533 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.457534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.458275 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.458277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.458658 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.458659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.458785 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.459283 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:36.459284 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:36.459306 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:36.459324 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:36.459476 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:36.459480 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:36.459482 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.459537 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.459636 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260923 22:11:36.460234 20437 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40679
I20260923 22:11:36.460311 20437 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:36.460431 20437 heartbeater.cc:511] Master 127.1.161.190:40679 requested a full tablet report, sending...
I20260923 22:11:36.460872 20249 ts_manager.cc:194] Registered new tserver with Master: 5758fe0ce5954b39a96adeeb407e4f70 (127.1.161.129:46565)
I20260923 22:11:36.461339 20249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.129:44447
I20260923 22:11:36.462922 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20309
I20260923 22:11:36.462999 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:36.463229 1670 external_mini_cluster.cc:1428] Setting key 64f21c303c3006df11447247cbcbacfe
I20260923 22:11:36.464026 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--builtin_ntp_servers=127.1.161.148:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:36.537678 20440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.537930 20440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.537968 20440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.539219 20440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.539274 20440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.539317 20440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:36.540606 20440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:36.540845 20440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.541126 20440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.545460 20448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.545495 20445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.545475 20446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.545699 20440 server_base.cc:1034] running on GCE node
I20260923 22:11:36.545861 20440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.546041 20440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.547310 20440 hybrid_clock.cc:648] HybridClock initialized: now 1790201496547300 us; error 24 us; skew 500 ppm
I20260923 22:11:36.551923 20440 webserver.cc:460] Webserver started at http://127.1.161.130:45593/ using document root <none> and password file <none>
I20260923 22:11:36.552137 20440 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.552181 20440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.552321 20440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.553282 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "b6eda243a7fb486388332d36da5b8733"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "3c7759370a13ea71a7a55beb02e9a8d0"
server_key_iv: "306e11c66f06324d0396b6212cea4fa0"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.553602 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "b6eda243a7fb486388332d36da5b8733"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "3c7759370a13ea71a7a55beb02e9a8d0"
server_key_iv: "306e11c66f06324d0396b6212cea4fa0"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.554811 20440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:36.555490 20454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.555640 20440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.555724 20440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "b6eda243a7fb486388332d36da5b8733"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "3c7759370a13ea71a7a55beb02e9a8d0"
server_key_iv: "306e11c66f06324d0396b6212cea4fa0"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.555792 20440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.561450 20440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.561672 20440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.561792 20440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.561978 20440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.562232 20440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.562278 20440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.562330 20440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.562356 20440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.567304 20440 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:36141
I20260923 22:11:36.567366 20567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:36141 every 8 connection(s)
I20260923 22:11:36.567641 20440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:36.571899 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.568314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:58149 (local address 127.1.161.190:40679)
0923 22:11:36.568370 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.568372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.568580 (+ 208us) server_negotiation.cc:407] Connection header received
0923 22:11:36.568985 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.568986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.568996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.569069 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.569621 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.570052 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.570852 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.571009 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.571070 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.571724 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.571725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.571726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:36.571729 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.571742 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:36.571745 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.571781 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.571830 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.571849 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:36.572108 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.568314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40679 (local address 127.1.161.130:58149)
0923 22:11:36.568472 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.568500 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.568909 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.569120 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.569122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.569130 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:36.569554 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.569557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.570120 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.570121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.570789 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.570791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.571082 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.571083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.571215 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.571598 (+ 383us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:36.571598 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:36.571621 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:36.571638 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:36.571787 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:36.571789 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:36.571791 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.571831 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.571894 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260923 22:11:36.572570 20568 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40679
I20260923 22:11:36.572643 20568 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:36.572829 20568 heartbeater.cc:511] Master 127.1.161.190:40679 requested a full tablet report, sending...
I20260923 22:11:36.572939 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20440
I20260923 22:11:36.573040 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:36.573174 20249 ts_manager.cc:194] Registered new tserver with Master: b6eda243a7fb486388332d36da5b8733 (127.1.161.130:36141)
I20260923 22:11:36.573227 1670 external_mini_cluster.cc:1428] Setting key 165d731d2039c05b8d8f71c128c382fa
I20260923 22:11:36.573704 20249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.130:58149
I20260923 22:11:36.574290 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--builtin_ntp_servers=127.1.161.148:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:36.648697 20571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:36.648885 20571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:36.648907 20571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:36.650161 20571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:36.650211 20571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:36.650236 20571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:36.651381 20571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:40679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:36.651594 20571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:36.651814 20571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:36.656275 20577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:36.656266 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.656389 20571 server_base.cc:1034] running on GCE node
W20260923 22:11:36.656267 20576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:36.656636 20571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:36.656859 20571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:36.658162 20571 hybrid_clock.cc:648] HybridClock initialized: now 1790201496658140 us; error 31 us; skew 500 ppm
I20260923 22:11:36.659363 20571 webserver.cc:460] Webserver started at http://127.1.161.131:41263/ using document root <none> and password file <none>
I20260923 22:11:36.659559 20571 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:36.659619 20571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:36.659759 20571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:36.660669 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "8c62f2fb8c8d45ff95a634c62c8942f1"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "b7a6b9b20998294f4b539c4d4db2ec5a"
server_key_iv: "9ee18fd86767dda3b88b6eda2f197f35"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.660959 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "8c62f2fb8c8d45ff95a634c62c8942f1"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "b7a6b9b20998294f4b539c4d4db2ec5a"
server_key_iv: "9ee18fd86767dda3b88b6eda2f197f35"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.662039 20571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:36.662660 20585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.662791 20571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:36.662870 20571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "8c62f2fb8c8d45ff95a634c62c8942f1"
format_stamp: "Formatted at 2026-09-23 22:11:36 on dist-test-slave-npvd"
server_key: "b7a6b9b20998294f4b539c4d4db2ec5a"
server_key_iv: "9ee18fd86767dda3b88b6eda2f197f35"
server_key_version: "encryptionkey@0"
I20260923 22:11:36.662942 20571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:36.670900 20571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:36.671134 20571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:36.671270 20571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:36.671458 20571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:36.671749 20571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:36.671804 20571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.671839 20571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:36.671864 20571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:36.677338 20571 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:43349
I20260923 22:11:36.677371 20698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:43349 every 8 connection(s)
I20260923 22:11:36.677663 20571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:36.678545 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20571
I20260923 22:11:36.678675 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:36.678866 1670 external_mini_cluster.cc:1428] Setting key 9d8c939823b203656179b6676798c670
I20260923 22:11:36.682468 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.678314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:50523 (local address 127.1.161.190:40679)
0923 22:11:36.678363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.678366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.678653 (+ 287us) server_negotiation.cc:407] Connection header received
0923 22:11:36.679292 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.679293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.679304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.679352 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.680080 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.680522 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681341 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.681515 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681564 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.682260 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.682260 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.682261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:36.682264 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.682275 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:36.682279 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.682316 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.682402 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.682418 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:36.682641 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.678304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:40679 (local address 127.1.161.131:50523)
0923 22:11:36.678521 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.678563 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.679157 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.679418 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.679422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.679431 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:36.680005 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.680010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.680590 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.680591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.681271 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.681273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.681581 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.681582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.681675 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.682150 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:36.682151 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:36.682174 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:36.682192 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:36.682330 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:36.682333 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:36.682335 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.682380 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.682446 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260923 22:11:36.683051 20699 heartbeater.cc:347] Connected to a master server at 127.1.161.190:40679
I20260923 22:11:36.683140 20699 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:36.683310 20699 heartbeater.cc:511] Master 127.1.161.190:40679 requested a full tablet report, sending...
I20260923 22:11:36.683626 20249 ts_manager.cc:194] Registered new tserver with Master: 8c62f2fb8c8d45ff95a634c62c8942f1 (127.1.161.131:43349)
I20260923 22:11:36.683938 20249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.131:50523
I20260923 22:11:36.690418 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:36.694015 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.691616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.1.161.190:40679)
0923 22:11:36.691667 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.691669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.691753 (+ 84us) server_negotiation.cc:407] Connection header received
0923 22:11:36.691811 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.691811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.691821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.691862 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:36.692006 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.692395 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.693046 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.693219 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.693278 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.693765 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:36.693766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:36.693766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:36.693770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:36.693780 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:36.693783 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:36.693879 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.693948 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.693957 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:36.695423 20249 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:36.696811 20249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:36.702602 20711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.699162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:43349 (local address 127.0.0.1:35590)
0923 22:11:36.699349 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.699357 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.699406 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.699615 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.699616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.699617 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.699645 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.699646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700670 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.700671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.701548 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.701550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702335 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.702411 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702424 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.702488 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.702537 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260923 22:11:36.702694 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.698897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.1.161.129:46565)
0923 22:11:36.699186 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.699191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.699202 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:11:36.699246 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.699248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.699262 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.699370 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.699562 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700551 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.701872 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702336 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702437 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702530 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.702568 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.702607 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260923 22:11:36.702742 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.699007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36141 (local address 127.0.0.1:50480)
0923 22:11:36.699191 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.699200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.699244 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.699677 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.699678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.699680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.699723 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.699725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700523 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.700524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.701751 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.701754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702559 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.702603 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702606 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.702631 (+ 25us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.702663 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":53,"thread_start_us":73,"threads_started":1}
I20260923 22:11:36.702811 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.698789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46565 (local address 127.0.0.1:33886)
0923 22:11:36.698937 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.698964 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.699054 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.699382 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.699396 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.699398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.699456 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.699457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700637 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.700638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.701751 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.701754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702444 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.702504 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702508 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.702544 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.702581 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":9,"thread_start_us":74,"threads_started":1}
I20260923 22:11:36.702843 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.699315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.1.161.131:43349)
0923 22:11:36.699453 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.699456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.699466 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:36.699493 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.699494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.699506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.699568 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.699706 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700587 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.701641 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702244 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702312 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702383 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.702711 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.702764 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260923 22:11:36.703068 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.699133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.1.161.130:36141)
0923 22:11:36.699521 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.699525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.699534 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:36.699569 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.699571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.699585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.699795 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.699856 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.700418 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.701838 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.702474 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.702849 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.702942 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.702965 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.703000 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":95,"threads_started":1}
I20260923 22:11:36.703423 20371 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9fe54614349a4924bb438bd15a0ea (DEFAULT_TABLE table=test-table [id=b93e2ecd68b0494cb957388ee05c410a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:36.703420 20633 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9fe54614349a4924bb438bd15a0ea (DEFAULT_TABLE table=test-table [id=b93e2ecd68b0494cb957388ee05c410a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:36.703711 20371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9fe54614349a4924bb438bd15a0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:36.703715 20633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9fe54614349a4924bb438bd15a0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:36.703696 20502 tablet_service.cc:1467] Processing CreateTablet for tablet b9a9fe54614349a4924bb438bd15a0ea (DEFAULT_TABLE table=test-table [id=b93e2ecd68b0494cb957388ee05c410a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:36.703966 20502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a9fe54614349a4924bb438bd15a0ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:36.705893 20713 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: Bootstrap starting.
I20260923 22:11:36.706885 20714 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Bootstrap starting.
I20260923 22:11:36.707101 20715 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Bootstrap starting.
I20260923 22:11:36.707458 20714 tablet_bootstrap.cc:654] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:36.707695 20715 tablet_bootstrap.cc:654] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:36.707870 20713 tablet_bootstrap.cc:654] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:36.708065 20715 log.cc:824] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:36.708192 20713 log.cc:824] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:36.708956 20713 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: No bootstrap required, opened a new log
I20260923 22:11:36.708976 20715 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: No bootstrap required, opened a new log
I20260923 22:11:36.709074 20715 ts_tablet_manager.cc:1397] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:36.709096 20714 log.cc:824] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:36.709118 20713 ts_tablet_manager.cc:1397] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 22:11:36.709784 20714 tablet_bootstrap.cc:492] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: No bootstrap required, opened a new log
I20260923 22:11:36.709849 20714 ts_tablet_manager.cc:1397] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 22:11:36.711047 20713 raft_consensus.cc:348] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.711205 20713 raft_consensus.cc:374] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:36.711251 20713 raft_consensus.cc:729] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5758fe0ce5954b39a96adeeb407e4f70, State: Initialized, Role: FOLLOWER
I20260923 22:11:36.711352 20713 consensus_queue.cc:260] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.711498 20715 raft_consensus.cc:348] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.711615 20713 ts_tablet_manager.cc:1428] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:36.711642 20715 raft_consensus.cc:374] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:36.711674 20715 raft_consensus.cc:729] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c62f2fb8c8d45ff95a634c62c8942f1, State: Initialized, Role: FOLLOWER
I20260923 22:11:36.711735 20437 heartbeater.cc:503] Master 127.1.161.190:40679 was elected leader, sending a full tablet report...
I20260923 22:11:36.711777 20715 consensus_queue.cc:260] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.712010 20715 ts_tablet_manager.cc:1428] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:36.712056 20714 raft_consensus.cc:348] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.712137 20699 heartbeater.cc:503] Master 127.1.161.190:40679 was elected leader, sending a full tablet report...
I20260923 22:11:36.712188 20714 raft_consensus.cc:374] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:36.712210 20714 raft_consensus.cc:729] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6eda243a7fb486388332d36da5b8733, State: Initialized, Role: FOLLOWER
I20260923 22:11:36.712303 20714 consensus_queue.cc:260] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.712525 20714 ts_tablet_manager.cc:1428] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:36.712596 20568 heartbeater.cc:503] Master 127.1.161.190:40679 was elected leader, sending a full tablet report...
I20260923 22:11:36.792392 20720 raft_consensus.cc:482] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:36.792493 20720 raft_consensus.cc:504] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.792794 20720 leader_election.cc:288] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5758fe0ce5954b39a96adeeb407e4f70 (127.1.161.129:46565), b6eda243a7fb486388332d36da5b8733 (127.1.161.130:36141)
I20260923 22:11:36.796156 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.793037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:47973 (local address 127.1.161.129:46565)
0923 22:11:36.793101 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.793103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.793112 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:36.793138 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.793139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.793147 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.793202 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.793454 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.794124 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795352 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.795815 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795881 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.795887 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.796044 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.796093 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:11:36.796154 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.793091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:56795 (local address 127.1.161.130:36141)
0923 22:11:36.793185 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.793187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.793389 (+ 202us) server_negotiation.cc:407] Connection header received
0923 22:11:36.793446 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.793447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.793456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.793512 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:36.793622 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.794192 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795350 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.795815 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795881 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.795887 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.796044 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.796093 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 22:11:36.796144 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.792926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46565 (local address 127.1.161.131:47973)
0923 22:11:36.792988 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.793000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.793051 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.793323 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.793324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.793325 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.793392 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.793394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.794193 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.794194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.795265 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.795268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.795888 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.795959 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.795962 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.796035 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.796081 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260923 22:11:36.796154 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.793045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:36141 (local address 127.1.161.131:56795)
0923 22:11:36.793322 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.793331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:36.793382 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:36.793524 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:36.793525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:36.793526 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:36.793563 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.793564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.794245 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.794245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.795275 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:36.795277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.795891 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.795892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:36.795965 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.795966 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:11:36.796013 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:11:36.796051 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":29,"thread_start_us":95,"threads_started":1}
I20260923 22:11:36.796533 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9fe54614349a4924bb438bd15a0ea" candidate_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5758fe0ce5954b39a96adeeb407e4f70" is_pre_election: true
I20260923 22:11:36.796540 20522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9fe54614349a4924bb438bd15a0ea" candidate_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6eda243a7fb486388332d36da5b8733" is_pre_election: true
I20260923 22:11:36.796669 20391 raft_consensus.cc:2454] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c62f2fb8c8d45ff95a634c62c8942f1 in term 0.
I20260923 22:11:36.796679 20522 raft_consensus.cc:2454] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c62f2fb8c8d45ff95a634c62c8942f1 in term 0.
I20260923 22:11:36.796898 20586 leader_election.cc:302] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c62f2fb8c8d45ff95a634c62c8942f1, b6eda243a7fb486388332d36da5b8733; no voters:
I20260923 22:11:36.797031 20720 raft_consensus.cc:2781] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:36.797107 20720 raft_consensus.cc:482] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:36.797139 20720 raft_consensus.cc:3037] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.797776 20720 raft_consensus.cc:504] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.797930 20720 leader_election.cc:288] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [CANDIDATE]: Term 1 election: Requested vote from peers 5758fe0ce5954b39a96adeeb407e4f70 (127.1.161.129:46565), b6eda243a7fb486388332d36da5b8733 (127.1.161.130:36141)
I20260923 22:11:36.798089 20391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9fe54614349a4924bb438bd15a0ea" candidate_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5758fe0ce5954b39a96adeeb407e4f70"
I20260923 22:11:36.798090 20522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a9fe54614349a4924bb438bd15a0ea" candidate_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6eda243a7fb486388332d36da5b8733"
I20260923 22:11:36.798182 20391 raft_consensus.cc:3037] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.798178 20522 raft_consensus.cc:3037] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:36.798862 20522 raft_consensus.cc:2454] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c62f2fb8c8d45ff95a634c62c8942f1 in term 1.
I20260923 22:11:36.798878 20391 raft_consensus.cc:2454] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c62f2fb8c8d45ff95a634c62c8942f1 in term 1.
I20260923 22:11:36.799042 20586 leader_election.cc:302] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5758fe0ce5954b39a96adeeb407e4f70, 8c62f2fb8c8d45ff95a634c62c8942f1; no voters:
I20260923 22:11:36.799155 20720 raft_consensus.cc:2781] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:36.799310 20720 raft_consensus.cc:686] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 1 LEADER]: Becoming Leader. State: Replica: 8c62f2fb8c8d45ff95a634c62c8942f1, State: Running, Role: LEADER
I20260923 22:11:36.799500 20720 consensus_queue.cc:237] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } }
I20260923 22:11:36.800170 20249 catalog_manager.cc:5495] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c62f2fb8c8d45ff95a634c62c8942f1 (127.1.161.131). New cstate: current_term: 1 leader_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c62f2fb8c8d45ff95a634c62c8942f1" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 43349 } health_report { overall_health: HEALTHY } } }
I20260923 22:11:36.816293 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.814066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.1.161.131:43349)
0923 22:11:36.814097 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:36.814099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:36.814112 (+ 13us) server_negotiation.cc:407] Connection header received
0923 22:11:36.814165 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:36.814166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:36.814178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:36.814227 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:36.814328 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.814757 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.815621 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:36.815808 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:36.815856 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:36.816037 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:36.816123 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:36.816187 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:36.816234 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:11:36.816240 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:11:36.817797 20391 raft_consensus.cc:1261] T b9a9fe54614349a4924bb438bd15a0ea P 5758fe0ce5954b39a96adeeb407e4f70 [term 1 FOLLOWER]: Refusing update from remote peer 8c62f2fb8c8d45ff95a634c62c8942f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:36.817797 20522 raft_consensus.cc:1261] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 1 FOLLOWER]: Refusing update from remote peer 8c62f2fb8c8d45ff95a634c62c8942f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260923 22:11:36.818148 20569 tablet.cc:2369] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:36.818169 20723 consensus_queue.cc:1035] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6eda243a7fb486388332d36da5b8733" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 36141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:36.818281 20720 consensus_queue.cc:1035] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5758fe0ce5954b39a96adeeb407e4f70" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 46565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:36.820561 20728 mvcc.cc:205] Tried to move back new op lower bound from 7332665330963374080 to 7332665330891333632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665330963374080}]
I20260923 22:11:36.820760 20727 mvcc.cc:205] Tried to move back new op lower bound from 7332665330963374080 to 7332665330891333632. Current Snapshot: MvccSnapshot[applied={T|T < 7332665330963374080}]
I20260923 22:11:36.822949 20249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:36.823045 20249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:36.823700 20249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40fad4ecbeab4ea39debc11882e6ad54: Sending DeleteTablet for 3 replicas of tablet b9a9fe54614349a4924bb438bd15a0ea
I20260923 22:11:36.823976 20371 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9fe54614349a4924bb438bd15a0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='slave'} at 127.0.0.1:33886
I20260923 22:11:36.824013 20502 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9fe54614349a4924bb438bd15a0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='slave'} at 127.0.0.1:50480
I20260923 22:11:36.824208 20737 tablet_replica.cc:330] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: stopping tablet replica
I20260923 22:11:36.824249 20633 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a9fe54614349a4924bb438bd15a0ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:36 UTC) from {username='slave'} at 127.0.0.1:35590
I20260923 22:11:36.824285 20737 raft_consensus.cc:2229] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:36.824358 20737 raft_consensus.cc:2258] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:36.824411 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20309
I20260923 22:11:36.824976 20737 ts_tablet_manager.cc:1905] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:36.826597 20737 ts_tablet_manager.cc:1918] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:36.826660 20737 log.cc:1196] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/b9a9fe54614349a4924bb438bd15a0ea
I20260923 22:11:36.826890 20737 ts_tablet_manager.cc:1939] T b9a9fe54614349a4924bb438bd15a0ea P b6eda243a7fb486388332d36da5b8733: Deleting consensus metadata
I20260923 22:11:36.827240 20233 catalog_manager.cc:4841] TS b6eda243a7fb486388332d36da5b8733 (127.1.161.130:36141): tablet b9a9fe54614349a4924bb438bd15a0ea (table test-table [id=b93e2ecd68b0494cb957388ee05c410a]) successfully deleted
I20260923 22:11:36.830220 20738 tablet_replica.cc:330] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: stopping tablet replica
W20260923 22:11:36.830225 20233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:46565 (error 108)
I20260923 22:11:36.830330 20738 raft_consensus.cc:2229] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:36.830381 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20440
I20260923 22:11:36.830516 20738 raft_consensus.cc:2258] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:36.830852 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:36.830551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:46565 (local address 127.0.0.1:33890)
0923 22:11:36.830795 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:36.830826 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
W20260923 22:11:36.830943 20233 catalog_manager.cc:4584] TS 5758fe0ce5954b39a96adeeb407e4f70 (127.1.161.129:46565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9a9fe54614349a4924bb438bd15a0ea: Network error: Client connection negotiation failed: client connection to 127.1.161.129:46565: connect: Connection refused (error 111)
I20260923 22:11:36.830976 20738 ts_tablet_manager.cc:1905] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:36.832778 20738 ts_tablet_manager.cc:1918] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:36.832847 20738 log.cc:1196] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/b9a9fe54614349a4924bb438bd15a0ea
I20260923 22:11:36.833055 20738 ts_tablet_manager.cc:1939] T b9a9fe54614349a4924bb438bd15a0ea P 8c62f2fb8c8d45ff95a634c62c8942f1: Deleting consensus metadata
I20260923 22:11:36.833389 20233 catalog_manager.cc:4841] TS 8c62f2fb8c8d45ff95a634c62c8942f1 (127.1.161.131:43349): tablet b9a9fe54614349a4924bb438bd15a0ea (table test-table [id=b93e2ecd68b0494cb957388ee05c410a]) successfully deleted
I20260923 22:11:36.835774 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20571
I20260923 22:11:36.839977 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20218
2026-09-23T22:11:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (661 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:36 dist-test-slave-npvd krb5kdc[20742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:36 dist-test-slave-npvd krb5kdc[20742](info): set up 2 sockets
Sep 23 22:11:36 dist-test-slave-npvd krb5kdc[20742](info): commencing operation
krb5kdc: starting...
W20260923 22:11:37.754391 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.896s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:37 dist-test-slave-npvd krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:37.854264 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36657
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:37.926656 20755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:37.926833 20755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:37.926852 20755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:37.927870 20755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:37.927927 20755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:37.927942 20755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:37.927954 20755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:37.928011 20755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:37.929400 20755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:37.929637 20755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:37.929836 20755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:37.933908 20761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:37.933902 20760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.934082 20755 server_base.cc:1034] running on GCE node
W20260923 22:11:37.933897 20763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:37.934387 20755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:37.934638 20755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:37.935787 20755 hybrid_clock.cc:648] HybridClock initialized: now 1790201497935770 us; error 38 us; skew 500 ppm
Sep 23 22:11:37 dist-test-slave-npvd krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201497, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:37.938093 20755 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:37.938493 20755 webserver.cc:460] Webserver started at http://127.1.161.190:42027/ using document root <none> and password file <none>
I20260923 22:11:37.938699 20755 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:37.938763 20755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:37.938906 20755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:37.939847 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "ead131943a5248089cdc9f5b9b93ab87"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-npvd"
server_key: "61900427c0925b4ecc78b1cb70bb14b4"
server_key_iv: "24cfc4a35f023f7847a1d198925eead1"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.940151 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "ead131943a5248089cdc9f5b9b93ab87"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-npvd"
server_key: "61900427c0925b4ecc78b1cb70bb14b4"
server_key_iv: "24cfc4a35f023f7847a1d198925eead1"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.941326 20755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:37.941982 20770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:37.942137 20755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:37.942225 20755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "ead131943a5248089cdc9f5b9b93ab87"
format_stamp: "Formatted at 2026-09-23 22:11:37 on dist-test-slave-npvd"
server_key: "61900427c0925b4ecc78b1cb70bb14b4"
server_key_iv: "24cfc4a35f023f7847a1d198925eead1"
server_key_version: "encryptionkey@0"
I20260923 22:11:37.942306 20755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:37.952581 20755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:37.954481 20755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:37.954617 20755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:37.958209 20755 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36657
I20260923 22:11:37.958240 20822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36657 every 8 connection(s)
I20260923 22:11:37.958546 20755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:37.959342 20823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:37.961179 20823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: Bootstrap starting.
I20260923 22:11:37.961685 20823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:37.961957 20823 log.cc:824] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:37.962563 20823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: No bootstrap required, opened a new log
I20260923 22:11:37.963908 20823 raft_consensus.cc:348] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } }
I20260923 22:11:37.964015 20823 raft_consensus.cc:374] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:37.964061 20823 raft_consensus.cc:729] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead131943a5248089cdc9f5b9b93ab87, State: Initialized, Role: FOLLOWER
I20260923 22:11:37.964187 20823 consensus_queue.cc:260] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } }
I20260923 22:11:37.964257 20823 raft_consensus.cc:388] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:37.964308 20823 raft_consensus.cc:482] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:37.964352 20823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:37.964809 20823 raft_consensus.cc:504] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } }
I20260923 22:11:37.964906 20823 leader_election.cc:302] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ead131943a5248089cdc9f5b9b93ab87; no voters:
I20260923 22:11:37.965076 20823 leader_election.cc:288] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:37.965106 20826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:37.965282 20823 sys_catalog.cc:564] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:37.965325 20826 raft_consensus.cc:686] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [term 1 LEADER]: Becoming Leader. State: Replica: ead131943a5248089cdc9f5b9b93ab87, State: Running, Role: LEADER
I20260923 22:11:37.965410 20826 consensus_queue.cc:237] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } }
I20260923 22:11:37.965819 20828 sys_catalog.cc:455] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ead131943a5248089cdc9f5b9b93ab87. Latest consensus state: current_term: 1 leader_uuid: "ead131943a5248089cdc9f5b9b93ab87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } } }
I20260923 22:11:37.965929 20828 sys_catalog.cc:458] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:37.965951 20827 sys_catalog.cc:455] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ead131943a5248089cdc9f5b9b93ab87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ead131943a5248089cdc9f5b9b93ab87" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36657 } } }
I20260923 22:11:37.966025 20827 sys_catalog.cc:458] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:37.966149 20835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:37.966697 20835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:37.967809 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20755
I20260923 22:11:37.967933 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:37.968138 1670 external_mini_cluster.cc:1428] Setting key 4bba2e0deab87164e6529be15a913e9e
I20260923 22:11:37.968255 20835 catalog_manager.cc:1347] Generated new cluster ID: 759184367bb34a7caef7130f576ef831
I20260923 22:11:37.968312 20835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:37.975025 20835 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:37.975613 20835 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 22:11:37 dist-test-slave-npvd krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:37.979439 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:37.968670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.1.161.190:36657)
0923 22:11:37.971282 (+ 2612us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:37.971289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:37.971316 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:11:37.971404 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:37.971409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:37.971441 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:37.971566 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:37.972099 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.972897 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.973586 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:37.974002 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:37.974122 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:37.976376 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:37.976379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:37.976380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:37.976389 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:37.977871 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.978235 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.978238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.978238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.978322 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:37.978520 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:37.978521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:37.978521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:37.978719 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:37.978878 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:37.978978 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 22:11:37.979289 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2568,"thread_start_us":72,"threads_started":1}
I20260923 22:11:37.981258 20835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: Generated new TSK 0
I20260923 22:11:37.981390 20835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.161.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
Entry for principal kudu/127.1.161.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab.
I20260923 22:11:38.008297 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:38.081699 20849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.081883 20849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.081914 20849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.082883 20849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.082940 20849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.082980 20849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:38.084522 20849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.129.keytab
--principal=kudu/127.1.161.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.084815 20849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.085024 20849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.089753 20854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.089785 20855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.089816 20849 server_base.cc:1034] running on GCE node
W20260923 22:11:38.089784 20857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.090134 20849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.090341 20849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.091518 20849 hybrid_clock.cc:648] HybridClock initialized: now 1790201498091511 us; error 34 us; skew 500 ppm
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:38.093641 20849 init.cc:377] Logged in from keytab as kudu/127.1.161.129@KRBTEST.COM (short username kudu)
I20260923 22:11:38.094022 20849 webserver.cc:460] Webserver started at http://127.1.161.129:38671/ using document root <none> and password file <none>
I20260923 22:11:38.094202 20849 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.094259 20849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.094376 20849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.095249 20849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "a4777ea3ed9245f182166814a5f3b17f"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "8c5699cce09ea57c4ab1c3e529466075"
server_key_iv: "ba73c381b06d426081a2f29227635694"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.095563 20849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "a4777ea3ed9245f182166814a5f3b17f"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "8c5699cce09ea57c4ab1c3e529466075"
server_key_iv: "ba73c381b06d426081a2f29227635694"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.096638 20849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:38.097301 20864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.097453 20849 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:38.097535 20849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "a4777ea3ed9245f182166814a5f3b17f"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "8c5699cce09ea57c4ab1c3e529466075"
server_key_iv: "ba73c381b06d426081a2f29227635694"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.097608 20849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.117704 20849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.119362 20849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.119495 20849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.119679 20849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:38.119922 20849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:38.119966 20849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.119993 20849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:38.120019 20849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.124867 20849 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:40555
I20260923 22:11:38.124903 20977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:40555 every 8 connection(s)
I20260923 22:11:38.125177 20849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.129@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:38.132189 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20849
I20260923 22:11:38.132287 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:38.132484 1670 external_mini_cluster.cc:1428] Setting key a67cb3e6cab48f56609be9cf036c4a5f
I20260923 22:11:38.133250 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.125783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:54817 (local address 127.1.161.190:36657)
0923 22:11:38.125837 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.125839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.126031 (+ 192us) server_negotiation.cc:407] Connection header received
0923 22:11:38.126416 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.126417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.126426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.126470 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.127267 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.127680 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.128579 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.128766 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.128815 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.130620 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.130621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.130622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:38.130628 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.131968 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.132308 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.132309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.132309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.132366 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.132602 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.132603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.132603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.132735 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.132798 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.133079 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.133157 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 22:11:38.133389 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.125779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36657 (local address 127.1.161.129:54817)
0923 22:11:38.125931 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.125962 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.126343 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.126532 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.126535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.126974 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:38.127135 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.127139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.127742 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.127743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.128507 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.128509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.128833 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.128834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.128921 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.129336 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:38.129343 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:38.130543 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:38.132035 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.132040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.132042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.132217 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.132430 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.132430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.132430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.132517 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.132794 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:38.132819 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:38.132952 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.133052 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.133207 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260923 22:11:38.133764 20978 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36657
I20260923 22:11:38.133836 20978 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:38.133975 20978 heartbeater.cc:511] Master 127.1.161.190:36657 requested a full tablet report, sending...
I20260923 22:11:38.134424 20787 ts_manager.cc:194] Registered new tserver with Master: a4777ea3ed9245f182166814a5f3b17f (127.1.161.129:40555)
I20260923 22:11:38.134965 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.129@KRBTEST.COM'} at 127.1.161.129:54817
WARNING: no policy specified for kudu/127.1.161.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
Entry for principal kudu/127.1.161.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab.
I20260923 22:11:38.154189 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:38.227707 20983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.228004 20983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.228080 20983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.229323 20983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.229382 20983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.229420 20983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:38.230724 20983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.130.keytab
--principal=kudu/127.1.161.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.230953 20983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.231151 20983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.235569 20991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.235579 20988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.235656 20983 server_base.cc:1034] running on GCE node
W20260923 22:11:38.235585 20989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.235982 20983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.236207 20983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.237366 20983 hybrid_clock.cc:648] HybridClock initialized: now 1790201498237341 us; error 25 us; skew 500 ppm
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:38.239789 20983 init.cc:377] Logged in from keytab as kudu/127.1.161.130@KRBTEST.COM (short username kudu)
I20260923 22:11:38.240115 20983 webserver.cc:460] Webserver started at http://127.1.161.130:35761/ using document root <none> and password file <none>
I20260923 22:11:38.240262 20983 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.240295 20983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.240370 20983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.241183 20983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "ddbcf6fb18134c3792f942f6ca0f57cc"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "45f3388d3ea409768b61cc8f95692827"
server_key_iv: "d0aa9c4bc7f41bc986276ffc994f1d4e"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.241442 20983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "ddbcf6fb18134c3792f942f6ca0f57cc"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "45f3388d3ea409768b61cc8f95692827"
server_key_iv: "d0aa9c4bc7f41bc986276ffc994f1d4e"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.242457 20983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:38.243024 20998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.243204 20983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.243296 20983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "ddbcf6fb18134c3792f942f6ca0f57cc"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "45f3388d3ea409768b61cc8f95692827"
server_key_iv: "d0aa9c4bc7f41bc986276ffc994f1d4e"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.243369 20983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.250979 20983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.252485 20983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.252619 20983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.252810 20983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:38.253064 20983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:38.253114 20983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.253162 20983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:38.253187 20983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.258561 20983 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:45541
I20260923 22:11:38.258637 21111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:45541 every 8 connection(s)
I20260923 22:11:38.258900 20983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.130@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:38.266360 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.259547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:49295 (local address 127.1.161.190:36657)
0923 22:11:38.259601 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.259603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.259776 (+ 173us) server_negotiation.cc:407] Connection header received
0923 22:11:38.260143 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.260144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.260151 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.260197 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.260892 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.261322 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.262085 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.262236 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.262278 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.263894 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.263895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.263895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:38.263900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.265103 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.265492 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.265493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.265493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.265535 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.265705 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.265706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.265706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.265779 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.265818 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.266202 (+ 384us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.266258 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 22:11:38.266594 21114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.259524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36657 (local address 127.1.161.130:49295)
0923 22:11:38.259669 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.259701 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.260072 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.260235 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.260238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.260678 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:38.260821 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.260824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.261391 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.261391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.262023 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.262024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.262297 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.262298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.262375 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.262696 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:38.262702 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:38.263820 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:38.265186 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.265191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.265193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.265437 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.265584 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.265585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.265585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.265664 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.265917 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:38.265924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:38.266069 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.266185 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.266367 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260923 22:11:38.267087 21112 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36657
I20260923 22:11:38.267158 21112 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:38.267364 21112 heartbeater.cc:511] Master 127.1.161.190:36657 requested a full tablet report, sending...
I20260923 22:11:38.267688 20787 ts_manager.cc:194] Registered new tserver with Master: ddbcf6fb18134c3792f942f6ca0f57cc (127.1.161.130:45541)
I20260923 22:11:38.267997 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 20983
I20260923 22:11:38.268110 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:38.268224 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.130@KRBTEST.COM'} at 127.1.161.130:49295
I20260923 22:11:38.268307 1670 external_mini_cluster.cc:1428] Setting key 6fd912a7148e235ca14be6a5bf43020d
WARNING: no policy specified for kudu/127.1.161.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
Entry for principal kudu/127.1.161.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab.
I20260923 22:11:38.290441 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:45465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:38.363209 21117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.363379 21117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.363400 21117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.364410 21117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.364446 21117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.364470 21117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:38.365829 21117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:45465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.131.keytab
--principal=kudu/127.1.161.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.365984 21117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.366154 21117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.370685 21123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.370666 21122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.370656 21125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.370853 21117 server_base.cc:1034] running on GCE node
I20260923 22:11:38.371042 21117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.371289 21117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.372452 21117 hybrid_clock.cc:648] HybridClock initialized: now 1790201498372447 us; error 35 us; skew 500 ppm
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:38.374716 21117 init.cc:377] Logged in from keytab as kudu/127.1.161.131@KRBTEST.COM (short username kudu)
I20260923 22:11:38.375219 21117 webserver.cc:460] Webserver started at http://127.1.161.131:40991/ using document root <none> and password file <none>
I20260923 22:11:38.375435 21117 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.375492 21117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.375612 21117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.376477 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "1280ccef972740239f9af4138c678c01"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d06c346cde2e21daaa87725f231d6e6e"
server_key_iv: "db29a6ad1daf3e96c1a6982f67dc5e6c"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.376806 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "1280ccef972740239f9af4138c678c01"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d06c346cde2e21daaa87725f231d6e6e"
server_key_iv: "db29a6ad1daf3e96c1a6982f67dc5e6c"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.378000 21117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:38.378695 21132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.378855 21117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:38.378952 21117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "1280ccef972740239f9af4138c678c01"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d06c346cde2e21daaa87725f231d6e6e"
server_key_iv: "db29a6ad1daf3e96c1a6982f67dc5e6c"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.379041 21117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.400278 21117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.401850 21117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.402000 21117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.402202 21117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:38.402458 21117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:38.402508 21117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.402536 21117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:38.402559 21117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.407678 21117 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:38057
I20260923 22:11:38.407727 21245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:38057 every 8 connection(s)
I20260923 22:11:38.408056 21117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
Sep 23 22:11:38 dist-test-slave-npvd krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201498, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.131@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:38.414383 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21117
I20260923 22:11:38.414505 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:38.414692 1670 external_mini_cluster.cc:1428] Setting key fa461e46f4040bf080ad587509374444
I20260923 22:11:38.415964 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.408753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:46157 (local address 127.1.161.190:36657)
0923 22:11:38.408802 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.408804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.408997 (+ 193us) server_negotiation.cc:407] Connection header received
0923 22:11:38.409429 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.409430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.409440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.409493 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.410275 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.410703 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.411484 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.411649 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.411698 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.413382 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.413383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.413384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:38.413389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.414692 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.415140 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.415141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.415141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.415216 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.415476 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.415477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.415478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.415618 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.415680 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.415858 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.415910 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:38.416172 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.408733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36657 (local address 127.1.161.131:46157)
0923 22:11:38.408887 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.408919 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.409324 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.409565 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.409568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.410037 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:38.410190 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.410193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.410773 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.410774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.411424 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.411426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.411713 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.411713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.411792 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.412138 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:11:38.412144 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:38.413303 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:38.414781 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.414786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.414788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.415062 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.415287 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:11:38.415287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:11:38.415288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:11:38.415402 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:11:38.415677 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:38.415681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:38.415774 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.415846 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.415943 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260923 22:11:38.416607 21246 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36657
I20260923 22:11:38.416705 21246 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:38.416880 21246 heartbeater.cc:511] Master 127.1.161.190:36657 requested a full tablet report, sending...
I20260923 22:11:38.417234 20787 ts_manager.cc:194] Registered new tserver with Master: 1280ccef972740239f9af4138c678c01 (127.1.161.131:38057)
I20260923 22:11:38.417521 20787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.161.131@KRBTEST.COM'} at 127.1.161.131:46157
I20260923 22:11:38.426381 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:38.432951 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.427735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.1.161.190:36657)
0923 22:11:38.427795 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.427797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.427885 (+ 88us) server_negotiation.cc:407] Connection header received
0923 22:11:38.427941 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.427942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.427952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.428063 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.428612 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.429056 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.429734 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.429888 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.429948 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.430980 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.430982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.430982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:38.430989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.431957 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.432276 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.432277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.432277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.432321 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:38.432502 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:38.432503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:38.432503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:38.432627 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.432718 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.432805 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.432871 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:11:38.434149 20787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:38.435587 20787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:38.440739 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:38057 (local address 127.0.0.1:38108)
0923 22:11:38.437347 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.437380 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.437430 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.437632 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.437635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.437638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.437691 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.437693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.438750 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.438751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.439606 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.439608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.440435 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.440436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.440533 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.440546 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.440619 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.440670 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"thread_start_us":45,"threads_started":1}
I20260923 22:11:38.440747 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.1.161.131:38057)
0923 22:11:38.437460 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.437464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.437473 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:11:38.437501 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.437503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.437521 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.437625 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.437755 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.438656 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.439668 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.440357 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.440432 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.440489 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.440641 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.440677 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260923 22:11:38.441438 21180 tablet_service.cc:1467] Processing CreateTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 (DEFAULT_TABLE table=test-table [id=883047ab28974af6933539456526e4a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:38.441584 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:45541 (local address 127.0.0.1:55316)
0923 22:11:38.437714 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.437723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.437761 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.438289 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.438290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.438292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.438329 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.438331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.439579 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.439580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.440566 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.440569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.441363 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.441364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.441415 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.441419 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.441483 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.441525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260923 22:11:38.441592 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.1.161.130:45541)
0923 22:11:38.438076 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.438080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.438090 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:38.438125 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.438127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.438142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.438215 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.438394 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.439486 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.440649 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.441287 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.441362 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.441404 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.441495 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.441528 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":125,"threads_started":1}
I20260923 22:11:38.441740 21180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87bec61a69d946ed8a89d5d77fe12935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:38.442184 21046 tablet_service.cc:1467] Processing CreateTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 (DEFAULT_TABLE table=test-table [id=883047ab28974af6933539456526e4a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:38.442394 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40555 (local address 127.0.0.1:53568)
0923 22:11:38.437546 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.437554 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.437589 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.438294 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.438294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.438296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.438343 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.438345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.439622 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.439623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.441055 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.441058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.442185 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.442186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.442243 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.442247 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.442303 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.442336 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":26,"thread_start_us":82,"threads_started":1}
I20260923 22:11:38.442518 21046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87bec61a69d946ed8a89d5d77fe12935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:38.442881 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.437898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.1.161.129:40555)
0923 22:11:38.438123 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.438127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.438137 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:38.438176 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.438178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.438193 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.438422 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.438513 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.439530 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.441173 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.442101 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.442173 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.442224 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.442777 (+ 553us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.442818 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":120,"threads_started":1}
I20260923 22:11:38.444334 21260 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Bootstrap starting.
I20260923 22:11:38.444888 20912 tablet_service.cc:1467] Processing CreateTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 (DEFAULT_TABLE table=test-table [id=883047ab28974af6933539456526e4a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:38.444967 21260 tablet_bootstrap.cc:654] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:38.445163 20912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87bec61a69d946ed8a89d5d77fe12935. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:38.445498 21260 log.cc:824] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:38.445891 21261 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Bootstrap starting.
I20260923 22:11:38.446255 21260 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: No bootstrap required, opened a new log
I20260923 22:11:38.446307 21260 ts_tablet_manager.cc:1397] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:38.447693 21261 tablet_bootstrap.cc:654] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:38.447741 21260 raft_consensus.cc:348] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.447846 21263 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: Bootstrap starting.
I20260923 22:11:38.447903 21260 raft_consensus.cc:374] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:38.447947 21260 raft_consensus.cc:729] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1280ccef972740239f9af4138c678c01, State: Initialized, Role: FOLLOWER
I20260923 22:11:38.448117 21260 consensus_queue.cc:260] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.448405 21260 ts_tablet_manager.cc:1428] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 22:11:38.448419 21246 heartbeater.cc:503] Master 127.1.161.190:36657 was elected leader, sending a full tablet report...
I20260923 22:11:38.448565 21263 tablet_bootstrap.cc:654] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:38.448355 21261 log.cc:824] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:38.449105 21263 log.cc:824] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:38.449354 21261 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: No bootstrap required, opened a new log
I20260923 22:11:38.449424 21261 ts_tablet_manager.cc:1397] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 22:11:38.449785 21263 tablet_bootstrap.cc:492] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: No bootstrap required, opened a new log
I20260923 22:11:38.449867 21263 ts_tablet_manager.cc:1397] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:38.450809 21261 raft_consensus.cc:348] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.450971 21261 raft_consensus.cc:374] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:38.450994 21261 raft_consensus.cc:729] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddbcf6fb18134c3792f942f6ca0f57cc, State: Initialized, Role: FOLLOWER
I20260923 22:11:38.451104 21261 consensus_queue.cc:260] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.451354 21261 ts_tablet_manager.cc:1428] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:38.451391 21112 heartbeater.cc:503] Master 127.1.161.190:36657 was elected leader, sending a full tablet report...
I20260923 22:11:38.451421 21263 raft_consensus.cc:348] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.451568 21263 raft_consensus.cc:374] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:38.451609 21263 raft_consensus.cc:729] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4777ea3ed9245f182166814a5f3b17f, State: Initialized, Role: FOLLOWER
I20260923 22:11:38.451704 21263 consensus_queue.cc:260] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.451921 21263 ts_tablet_manager.cc:1428] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 22:11:38.451938 20978 heartbeater.cc:503] Master 127.1.161.190:36657 was elected leader, sending a full tablet report...
I20260923 22:11:38.493974 21268 raft_consensus.cc:482] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:38.494068 21268 raft_consensus.cc:504] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.494354 21268 leader_election.cc:288] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1280ccef972740239f9af4138c678c01 (127.1.161.131:38057), ddbcf6fb18134c3792f942f6ca0f57cc (127.1.161.130:45541)
I20260923 22:11:38.497972 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.494590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:56231 (local address 127.1.161.131:38057)
0923 22:11:38.494641 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.494643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.494651 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:38.494674 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.494675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.494683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.494754 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.494913 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.495583 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497033 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.497546 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497616 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.497623 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.497873 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.497908 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 22:11:38.497972 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.494706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:45107 (local address 127.1.161.130:45541)
0923 22:11:38.494761 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.494762 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.494837 (+ 75us) server_negotiation.cc:407] Connection header received
0923 22:11:38.494904 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.494905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.494913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.494961 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:38.495084 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.495710 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497031 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.497554 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497616 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.497623 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.497873 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.497908 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:11:38.498131 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.494498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:38057 (local address 127.1.161.129:56231)
0923 22:11:38.494550 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.494562 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.494600 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.494776 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.494778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.494780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.494850 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.494852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.495684 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.495685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.496946 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.496949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.497629 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.497775 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.497776 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.497856 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.497901 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":126}
I20260923 22:11:38.498348 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.494635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:45541 (local address 127.1.161.129:45107)
0923 22:11:38.494771 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.494779 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.494832 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.494980 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.494980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.494982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:38.495024 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.495025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.495786 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.495786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.496946 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.496949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.497629 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.497630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.497771 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.497775 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.497855 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.497900 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":34,"thread_start_us":84,"threads_started":1}
I20260923 22:11:38.498528 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87bec61a69d946ed8a89d5d77fe12935" candidate_uuid: "a4777ea3ed9245f182166814a5f3b17f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1280ccef972740239f9af4138c678c01" is_pre_election: true
I20260923 22:11:38.498672 21200 raft_consensus.cc:2454] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4777ea3ed9245f182166814a5f3b17f in term 0.
I20260923 22:11:38.498684 21066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87bec61a69d946ed8a89d5d77fe12935" candidate_uuid: "a4777ea3ed9245f182166814a5f3b17f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" is_pre_election: true
I20260923 22:11:38.498790 21066 raft_consensus.cc:2454] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4777ea3ed9245f182166814a5f3b17f in term 0.
I20260923 22:11:38.498903 20867 leader_election.cc:302] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1280ccef972740239f9af4138c678c01, a4777ea3ed9245f182166814a5f3b17f; no voters:
I20260923 22:11:38.499092 21268 raft_consensus.cc:2781] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:38.499177 21268 raft_consensus.cc:482] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:38.499231 21268 raft_consensus.cc:3037] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.499863 21268 raft_consensus.cc:504] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.499992 21268 leader_election.cc:288] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [CANDIDATE]: Term 1 election: Requested vote from peers 1280ccef972740239f9af4138c678c01 (127.1.161.131:38057), ddbcf6fb18134c3792f942f6ca0f57cc (127.1.161.130:45541)
I20260923 22:11:38.500138 21200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87bec61a69d946ed8a89d5d77fe12935" candidate_uuid: "a4777ea3ed9245f182166814a5f3b17f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1280ccef972740239f9af4138c678c01"
I20260923 22:11:38.500156 21066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87bec61a69d946ed8a89d5d77fe12935" candidate_uuid: "a4777ea3ed9245f182166814a5f3b17f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc"
I20260923 22:11:38.500247 21066 raft_consensus.cc:3037] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.500250 21200 raft_consensus.cc:3037] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.500900 21066 raft_consensus.cc:2454] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4777ea3ed9245f182166814a5f3b17f in term 1.
I20260923 22:11:38.501088 20867 leader_election.cc:302] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4777ea3ed9245f182166814a5f3b17f, ddbcf6fb18134c3792f942f6ca0f57cc; no voters:
I20260923 22:11:38.501123 21200 raft_consensus.cc:2454] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4777ea3ed9245f182166814a5f3b17f in term 1.
I20260923 22:11:38.501204 21268 raft_consensus.cc:2781] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:38.501362 21268 raft_consensus.cc:686] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [term 1 LEADER]: Becoming Leader. State: Replica: a4777ea3ed9245f182166814a5f3b17f, State: Running, Role: LEADER
I20260923 22:11:38.501456 21268 consensus_queue.cc:237] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } }
I20260923 22:11:38.502089 20787 catalog_manager.cc:5495] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f reported cstate change: term changed from 0 to 1, leader changed from <none> to a4777ea3ed9245f182166814a5f3b17f (127.1.161.129). New cstate: current_term: 1 leader_uuid: "a4777ea3ed9245f182166814a5f3b17f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4777ea3ed9245f182166814a5f3b17f" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 40555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 } health_report { overall_health: UNKNOWN } } }
W20260923 22:11:38.509440 21113 tablet.cc:2369] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:38.515030 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.512876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.1.161.129:40555)
0923 22:11:38.512944 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.512946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.512954 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:38.512987 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.512987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.513021 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.513096 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:38.513214 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.513655 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.514312 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.514481 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.514562 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.514723 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:38.514814 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:38.514907 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.514923 (+ 16us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.514939 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 22:11:38.516530 21066 raft_consensus.cc:1261] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 1 FOLLOWER]: Refusing update from remote peer a4777ea3ed9245f182166814a5f3b17f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:38.516587 21200 raft_consensus.cc:1261] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 1 FOLLOWER]: Refusing update from remote peer a4777ea3ed9245f182166814a5f3b17f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:38.516855 21268 consensus_queue.cc:1035] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddbcf6fb18134c3792f942f6ca0f57cc" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 45541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:38.516995 21270 consensus_queue.cc:1035] T 87bec61a69d946ed8a89d5d77fe12935 P a4777ea3ed9245f182166814a5f3b17f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1280ccef972740239f9af4138c678c01" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:38.519647 21275 mvcc.cc:205] Tried to move back new op lower bound from 7332665337921413120 to 7332665337862549504. Current Snapshot: MvccSnapshot[applied={T|T < 7332665337921413120}]
I20260923 22:11:38.521210 20787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:38.521296 20787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:38.522037 20787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ead131943a5248089cdc9f5b9b93ab87: Sending DeleteTablet for 3 replicas of tablet 87bec61a69d946ed8a89d5d77fe12935
I20260923 22:11:38.522400 20912 tablet_service.cc:1514] Processing DeleteTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:53568
I20260923 22:11:38.522449 21046 tablet_service.cc:1514] Processing DeleteTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:55316
I20260923 22:11:38.522603 21180 tablet_service.cc:1514] Processing DeleteTablet for tablet 87bec61a69d946ed8a89d5d77fe12935 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:38 UTC) from {username='kudu', principal='kudu/127.1.161.190@KRBTEST.COM'} at 127.0.0.1:38108
I20260923 22:11:38.522691 21283 tablet_replica.cc:330] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: stopping tablet replica
I20260923 22:11:38.522770 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20849
I20260923 22:11:38.523403 21284 tablet_replica.cc:330] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: stopping tablet replica
I20260923 22:11:38.523483 21284 raft_consensus.cc:2229] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:38.523502 21283 raft_consensus.cc:2229] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:38.523561 21284 raft_consensus.cc:2258] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:38.523607 21283 raft_consensus.cc:2258] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:38.523973 21283 ts_tablet_manager.cc:1905] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:38.524412 21284 ts_tablet_manager.cc:1905] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:38.525681 21284 ts_tablet_manager.cc:1918] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:38.525772 21284 log.cc:1196] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/87bec61a69d946ed8a89d5d77fe12935
I20260923 22:11:38.525825 21283 ts_tablet_manager.cc:1918] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:38.525902 21283 log.cc:1196] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/87bec61a69d946ed8a89d5d77fe12935
I20260923 22:11:38.526057 21284 ts_tablet_manager.cc:1939] T 87bec61a69d946ed8a89d5d77fe12935 P 1280ccef972740239f9af4138c678c01: Deleting consensus metadata
I20260923 22:11:38.526229 21283 ts_tablet_manager.cc:1939] T 87bec61a69d946ed8a89d5d77fe12935 P ddbcf6fb18134c3792f942f6ca0f57cc: Deleting consensus metadata
I20260923 22:11:38.526410 20773 catalog_manager.cc:4841] TS 1280ccef972740239f9af4138c678c01 (127.1.161.131:38057): tablet 87bec61a69d946ed8a89d5d77fe12935 (table test-table [id=883047ab28974af6933539456526e4a1]) successfully deleted
I20260923 22:11:38.526572 20773 catalog_manager.cc:4841] TS ddbcf6fb18134c3792f942f6ca0f57cc (127.1.161.130:45541): tablet 87bec61a69d946ed8a89d5d77fe12935 (table test-table [id=883047ab28974af6933539456526e4a1]) successfully deleted
W20260923 22:11:38.529049 20773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:40555 (error 108)
I20260923 22:11:38.529227 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20983
I20260923 22:11:38.529738 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.529364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:40555 (local address 127.0.0.1:53600)
0923 22:11:38.529669 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.529699 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290}
W20260923 22:11:38.530038 20773 catalog_manager.cc:4584] TS a4777ea3ed9245f182166814a5f3b17f (127.1.161.129:40555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87bec61a69d946ed8a89d5d77fe12935: Network error: Client connection negotiation failed: client connection to 127.1.161.129:40555: connect: Connection refused (error 111)
I20260923 22:11:38.533819 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21117
I20260923 22:11:38.538028 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 20755
2026-09-23T22:11:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1700 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T22:11:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:38Z Disabled control of system clock
I20260923 22:11:38.564842 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36201
--webserver_interface=127.1.161.190
--webserver_port=0
--builtin_ntp_servers=127.1.161.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:38.641460 21289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.641655 21289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.641674 21289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.642777 21289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:38.642815 21289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.642827 21289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.642840 21289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:38.642853 21289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:38.644207 21289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.644392 21289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.644610 21289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.649415 21294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.649425 21297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.649425 21295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.649487 21289 server_base.cc:1034] running on GCE node
I20260923 22:11:38.649922 21289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.650169 21289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.651291 21289 hybrid_clock.cc:648] HybridClock initialized: now 1790201498651279 us; error 33 us; skew 500 ppm
I20260923 22:11:38.652392 21289 webserver.cc:460] Webserver started at http://127.1.161.190:35849/ using document root <none> and password file <none>
I20260923 22:11:38.652585 21289 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.652623 21289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.652704 21289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.653616 21289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "21b85b48790e4285aa66e1bd4b496d71"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d9b8fd2c4c3a8a1b6e9703b6bbe5947b"
server_key_iv: "272b0795e23a95cb8e1b88d10ca1d900"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.653944 21289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "21b85b48790e4285aa66e1bd4b496d71"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d9b8fd2c4c3a8a1b6e9703b6bbe5947b"
server_key_iv: "272b0795e23a95cb8e1b88d10ca1d900"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.655071 21289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:38.655757 21303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.655921 21289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.656000 21289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "21b85b48790e4285aa66e1bd4b496d71"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "d9b8fd2c4c3a8a1b6e9703b6bbe5947b"
server_key_iv: "272b0795e23a95cb8e1b88d10ca1d900"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.656064 21289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.685931 21289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.686195 21289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.686293 21289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.690050 21289 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36201
I20260923 22:11:38.690104 21355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36201 every 8 connection(s)
I20260923 22:11:38.690409 21289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:38.691356 21356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:38.693502 21356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: Bootstrap starting.
I20260923 22:11:38.694049 21356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:38.694342 21356 log.cc:824] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:38.695022 21356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: No bootstrap required, opened a new log
I20260923 22:11:38.696605 21356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } }
I20260923 22:11:38.696789 21356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:38.696835 21356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b85b48790e4285aa66e1bd4b496d71, State: Initialized, Role: FOLLOWER
I20260923 22:11:38.697000 21356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } }
I20260923 22:11:38.697079 21356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:38.697124 21356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:38.697170 21356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:38.697654 21356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } }
I20260923 22:11:38.697769 21356 leader_election.cc:302] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21b85b48790e4285aa66e1bd4b496d71; no voters:
I20260923 22:11:38.697947 21356 leader_election.cc:288] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:38.697993 21359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:38.698192 21356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:38.698199 21359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [term 1 LEADER]: Becoming Leader. State: Replica: 21b85b48790e4285aa66e1bd4b496d71, State: Running, Role: LEADER
I20260923 22:11:38.698391 21359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } }
I20260923 22:11:38.698731 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21289
I20260923 22:11:38.698861 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:38.698895 21360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21b85b48790e4285aa66e1bd4b496d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } } }
I20260923 22:11:38.698992 21360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:38.699054 1670 external_mini_cluster.cc:1428] Setting key f392d7066610a03144bd299c91cfbe51
I20260923 22:11:38.699615 21368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:38.699787 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b85b48790e4285aa66e1bd4b496d71. Latest consensus state: current_term: 1 leader_uuid: "21b85b48790e4285aa66e1bd4b496d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b85b48790e4285aa66e1bd4b496d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36201 } } }
I20260923 22:11:38.699865 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:38.700312 21368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:38.702512 21368 catalog_manager.cc:1347] Generated new cluster ID: 516b760ddb334462b57564786d8a6303
I20260923 22:11:38.702559 21368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:38.704535 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.699536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.1.161.190:36201)
0923 22:11:38.699699 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.699709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.700049 (+ 340us) server_negotiation.cc:407] Connection header received
0923 22:11:38.700163 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.700170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.700242 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.700391 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.700620 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.701653 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.702413 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.702995 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.703250 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.703843 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.703846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.703846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:38.704030 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.704076 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:38.704089 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.704147 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.704273 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.704343 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260923 22:11:38.709887 21368 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:38.710330 21368 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:38.714594 21368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: Generated new TSK 0
I20260923 22:11:38.714753 21368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:38.718715 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.129:0
--local_ip_for_outbound_sockets=127.1.161.129
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--builtin_ntp_servers=127.1.161.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:38.797811 21380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.798102 21380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.798141 21380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.799479 21380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.799528 21380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.799568 21380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.129
I20260923 22:11:38.800824 21380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.161.129
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.129
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.801041 21380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.801291 21380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.805701 21386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.805722 21385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.805858 21388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.806023 21380 server_base.cc:1034] running on GCE node
I20260923 22:11:38.806176 21380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.806394 21380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.807528 21380 hybrid_clock.cc:648] HybridClock initialized: now 1790201498807512 us; error 30 us; skew 500 ppm
I20260923 22:11:38.808710 21380 webserver.cc:460] Webserver started at http://127.1.161.129:38739/ using document root <none> and password file <none>
I20260923 22:11:38.808941 21380 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.808998 21380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.809134 21380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.810019 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data/instance:
uuid: "dcf7b92c2b2d44ddba03834d49cdca26"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "44dfbe28fcd1451ec3582ec6de807487"
server_key_iv: "117775d667cb81ad6234966542d4b0a2"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.810302 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf7b92c2b2d44ddba03834d49cdca26"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "44dfbe28fcd1451ec3582ec6de807487"
server_key_iv: "117775d667cb81ad6234966542d4b0a2"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.811482 21380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:38.812304 21394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.812475 21380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:38.812575 21380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal
uuid: "dcf7b92c2b2d44ddba03834d49cdca26"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "44dfbe28fcd1451ec3582ec6de807487"
server_key_iv: "117775d667cb81ad6234966542d4b0a2"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.812660 21380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.824595 21380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.824817 21380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.824940 21380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.825116 21380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:38.825397 21380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:38.825440 21380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.825474 21380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:38.825497 21380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.830657 21380 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.129:39629
I20260923 22:11:38.830696 21507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.129:39629 every 8 connection(s)
I20260923 22:11:38.830940 21380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/data/info.pb
I20260923 22:11:38.832945 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21380
I20260923 22:11:38.833066 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-0/wal/instance
I20260923 22:11:38.833223 1670 external_mini_cluster.cc:1428] Setting key 6ef59402d6fb6f34e97204ecf4aa5ead
I20260923 22:11:38.834098 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.130:0
--local_ip_for_outbound_sockets=127.1.161.130
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--builtin_ntp_servers=127.1.161.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 22:11:38.835749 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.831491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.129:60757 (local address 127.1.161.190:36201)
0923 22:11:38.831559 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.831561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.831784 (+ 223us) server_negotiation.cc:407] Connection header received
0923 22:11:38.832148 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.832149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.832159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.832233 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.832805 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.833371 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.834431 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.834717 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.834785 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.835552 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.835553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.835553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:38.835556 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.835569 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:38.835572 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.835608 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.835681 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.835702 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 22:11:38.835959 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.831483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36201 (local address 127.1.161.129:60757)
0923 22:11:38.831660 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.831686 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.832076 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.832277 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.832279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.832286 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:38.832728 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.832731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.833435 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.833436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.834354 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.834356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.834794 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.834795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.834888 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.835406 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:38.835407 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:38.835439 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:38.835464 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:38.835616 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:38.835619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:38.835622 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.835676 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.835764 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260923 22:11:38.836320 21508 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36201
I20260923 22:11:38.836395 21508 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:38.836555 21508 heartbeater.cc:511] Master 127.1.161.190:36201 requested a full tablet report, sending...
I20260923 22:11:38.837023 21320 ts_manager.cc:194] Registered new tserver with Master: dcf7b92c2b2d44ddba03834d49cdca26 (127.1.161.129:39629)
I20260923 22:11:38.837550 21320 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.129:60757
W20260923 22:11:38.904925 21511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:38.905117 21511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:38.905148 21511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:38.906100 21511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:38.906155 21511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:38.906190 21511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.130
I20260923 22:11:38.907367 21511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.161.130
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.130
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:38.907584 21511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:38.907820 21511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:38.912278 21516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:38.912307 21519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.912356 21511 server_base.cc:1034] running on GCE node
W20260923 22:11:38.912278 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:38.912734 21511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:38.912968 21511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:38.914136 21511 hybrid_clock.cc:648] HybridClock initialized: now 1790201498914122 us; error 43 us; skew 500 ppm
I20260923 22:11:38.915266 21511 webserver.cc:460] Webserver started at http://127.1.161.130:39537/ using document root <none> and password file <none>
I20260923 22:11:38.915480 21511 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:38.915541 21511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:38.915663 21511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:38.916631 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data/instance:
uuid: "c6b436924fc6412b86109c37e1dd71a3"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "b1f4a122d9f11e066fb525227719f02d"
server_key_iv: "2760c4a07924d43e37f235416c3d3d92"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.917021 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance:
uuid: "c6b436924fc6412b86109c37e1dd71a3"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "b1f4a122d9f11e066fb525227719f02d"
server_key_iv: "2760c4a07924d43e37f235416c3d3d92"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.918248 21511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:38.918989 21525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.919255 21511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:38.919342 21511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal
uuid: "c6b436924fc6412b86109c37e1dd71a3"
format_stamp: "Formatted at 2026-09-23 22:11:38 on dist-test-slave-npvd"
server_key: "b1f4a122d9f11e066fb525227719f02d"
server_key_iv: "2760c4a07924d43e37f235416c3d3d92"
server_key_version: "encryptionkey@0"
I20260923 22:11:38.919420 21511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:38.934896 21511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:38.935128 21511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:38.935274 21511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:38.935460 21511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:38.935722 21511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:38.935770 21511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.935798 21511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:38.935820 21511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:38.941298 21511 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.130:43351
I20260923 22:11:38.941350 21638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.130:43351 every 8 connection(s)
I20260923 22:11:38.941660 21511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/data/info.pb
I20260923 22:11:38.946596 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.942397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.130:44295 (local address 127.1.161.190:36201)
0923 22:11:38.942496 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:38.942498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:38.942692 (+ 194us) server_negotiation.cc:407] Connection header received
0923 22:11:38.943230 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:38.943231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:38.943246 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:38.943304 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:38.943959 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.944392 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.945330 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.945507 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.945574 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.946326 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:38.946327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:38.946328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:38.946332 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:38.946346 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:38.946351 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:38.946403 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:38.946496 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:11:38.946525 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 22:11:38.946774 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:38.942346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36201 (local address 127.1.161.130:44295)
0923 22:11:38.942556 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:38.942604 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:38.943094 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:38.943373 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:38.943377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:38.943385 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:38.943858 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.943861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.944489 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.944490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.945229 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:38.945231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:38.945592 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:38.945593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:38.945705 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:38.946191 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:38.946192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:38.946215 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:38.946234 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:38.946407 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:38.946411 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:38.946414 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:38.946498 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 22:11:38.946584 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260923 22:11:38.947222 21639 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36201
I20260923 22:11:38.947302 21639 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:38.947417 21639 heartbeater.cc:511] Master 127.1.161.190:36201 requested a full tablet report, sending...
I20260923 22:11:38.947702 21320 ts_manager.cc:194] Registered new tserver with Master: c6b436924fc6412b86109c37e1dd71a3 (127.1.161.130:43351)
I20260923 22:11:38.948062 21320 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.130:44295
I20260923 22:11:38.948887 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21511
I20260923 22:11:38.948999 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal/instance
I20260923 22:11:38.949194 1670 external_mini_cluster.cc:1428] Setting key 9bde8b08f3db342c459f0f085d33da07
I20260923 22:11:38.949911 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.161.131:0
--local_ip_for_outbound_sockets=127.1.161.131
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--builtin_ntp_servers=127.1.161.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 22:11:39.023146 21642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:39.023321 21642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:39.023340 21642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:39.024258 21642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:39.024293 21642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:39.024317 21642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.161.131
I20260923 22:11:39.025624 21642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.161.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.161.131
--webserver_port=0
--tserver_master_addrs=127.1.161.190:36201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.161.131
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:39.025784 21642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:39.025988 21642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:11:39.030400 21642 server_base.cc:1034] running on GCE node
W20260923 22:11:39.030419 21647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:39.030485 21650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:39.030422 21648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:39.030779 21642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:39.031042 21642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:39.032199 21642 hybrid_clock.cc:648] HybridClock initialized: now 1790201499032188 us; error 21 us; skew 500 ppm
I20260923 22:11:39.033267 21642 webserver.cc:460] Webserver started at http://127.1.161.131:34249/ using document root <none> and password file <none>
I20260923 22:11:39.033479 21642 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:39.033542 21642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:39.033667 21642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:39.034629 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data/instance:
uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-npvd"
server_key: "c01121df0fe52baf6b0a9ba79fa7385b"
server_key_iv: "d491c728f3c839a084502131817faca2"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.035009 21642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance:
uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-npvd"
server_key: "c01121df0fe52baf6b0a9ba79fa7385b"
server_key_iv: "d491c728f3c839a084502131817faca2"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.036309 21642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:39.036984 21656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:39.037169 21642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:39.037266 21642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal
uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a"
format_stamp: "Formatted at 2026-09-23 22:11:39 on dist-test-slave-npvd"
server_key: "c01121df0fe52baf6b0a9ba79fa7385b"
server_key_iv: "d491c728f3c839a084502131817faca2"
server_key_version: "encryptionkey@0"
I20260923 22:11:39.037349 21642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:39.054297 21642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:39.054533 21642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:39.054667 21642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:39.054862 21642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 22:11:39.055150 21642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 22:11:39.055262 21642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:39.055305 21642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 22:11:39.055331 21642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:39.061321 21642 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.131:41257
I20260923 22:11:39.061352 21769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.131:41257 every 8 connection(s)
I20260923 22:11:39.061621 21642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/data/info.pb
I20260923 22:11:39.064185 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21642
I20260923 22:11:39.064297 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal/instance
I20260923 22:11:39.064491 1670 external_mini_cluster.cc:1428] Setting key ea3b0bf525cf01854120b18db58d1271
I20260923 22:11:39.066166 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.062242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:41713 (local address 127.1.161.190:36201)
0923 22:11:39.062302 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.062304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.062483 (+ 179us) server_negotiation.cc:407] Connection header received
0923 22:11:39.062857 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.062858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.062867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.062920 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:39.063497 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.063917 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.064983 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.065230 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.065306 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.065987 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:39.065988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:39.065988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:39.065992 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:39.066005 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:39.066008 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:39.066045 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.066097 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.066118 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 22:11:39.066331 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.062202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:36201 (local address 127.1.161.131:41713)
0923 22:11:39.062353 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.062401 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.062789 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.062964 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.062967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.062973 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:11:39.063426 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.063430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.064008 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.064009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.064907 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.064910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.065330 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.065331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.065478 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.065871 (+ 393us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 22:11:39.065871 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:11:39.065894 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 22:11:39.065913 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:11:39.066046 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:11:39.066048 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:11:39.066050 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.066097 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.066160 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260923 22:11:39.066723 21770 heartbeater.cc:347] Connected to a master server at 127.1.161.190:36201
I20260923 22:11:39.066798 21770 heartbeater.cc:464] Registering TS with master...
I20260923 22:11:39.066960 21770 heartbeater.cc:511] Master 127.1.161.190:36201 requested a full tablet report, sending...
I20260923 22:11:39.067270 21320 ts_manager.cc:194] Registered new tserver with Master: 87a3b84a2c7d4e58ad55c61f4d789d4a (127.1.161.131:41257)
I20260923 22:11:39.067619 21320 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.161.131:41713
I20260923 22:11:39.076249 1670 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 22:11:39.080041 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.077478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.1.161.190:36201)
0923 22:11:39.077520 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.077522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.077660 (+ 138us) server_negotiation.cc:407] Connection header received
0923 22:11:39.077680 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.077681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.077692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.077744 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:39.077852 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.078293 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.078970 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.079136 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.079289 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.079838 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:39.079839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:39.079839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:39.079843 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:39.079851 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:39.079854 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:39.079897 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.079930 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.079949 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:39.081576 21320 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 22:11:39.082970 21320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 22:11:39.087945 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.084715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:43351 (local address 127.0.0.1:58544)
0923 22:11:39.084807 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.084837 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.084885 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.085203 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.085206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.085208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:39.085261 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.085262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086170 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.086170 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.086946 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.086947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.087749 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.087750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.087819 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.087827 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.087856 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.087887 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":55,"threads_started":1}
I20260923 22:11:39.088475 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.084801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58544 (local address 127.1.161.130:43351)
0923 22:11:39.085018 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.085021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.085029 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:11:39.085075 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.085077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.085090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.085318 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:39.085359 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086087 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.087032 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.087660 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.088204 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.088301 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.088323 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.088383 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260923 22:11:39.088611 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.084882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:39629 (local address 127.0.0.1:54450)
0923 22:11:39.085002 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.085010 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.085044 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.085475 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.085476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.085478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:39.085508 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.085509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086744 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.086744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.087468 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.087470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.088391 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.088392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.088453 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.088456 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.088498 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.088539 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":85,"threads_started":1}
I20260923 22:11:39.088649 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.085184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.1.161.129:39629)
0923 22:11:39.085317 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.085321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.085331 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:11:39.085367 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.085369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.085383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.085466 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:39.085603 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086638 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.087555 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.088288 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.088369 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.088472 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.088510 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.088561 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260923 22:11:39.088801 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.085057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.131:41257 (local address 127.0.0.1:37344)
0923 22:11:39.085145 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.085159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.085193 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.085492 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.085493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.085495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:39.085538 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.085540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086725 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.086725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.087672 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.087675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.088571 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.088572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.088634 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.088639 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.088675 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.088720 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":6,"thread_start_us":57,"threads_started":1}
I20260923 22:11:39.089099 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.085126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.1.161.131:41257)
0923 22:11:39.085360 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.085363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.085370 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:11:39.085395 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.085397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.085410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.085601 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:39.085640 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.086635 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.087732 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.088461 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.088901 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.088999 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.089021 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.089046 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":55,"threads_started":1}
I20260923 22:11:39.089100 21573 tablet_service.cc:1467] Processing CreateTablet for tablet 60e80766b0f547f49c613b833d78df38 (DEFAULT_TABLE table=test-table [id=776924b585f84030b7986c5cf0ce8eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:39.089262 21442 tablet_service.cc:1467] Processing CreateTablet for tablet 60e80766b0f547f49c613b833d78df38 (DEFAULT_TABLE table=test-table [id=776924b585f84030b7986c5cf0ce8eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:39.089434 21573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e80766b0f547f49c613b833d78df38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:39.089545 21442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e80766b0f547f49c613b833d78df38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:39.092432 21785 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: Bootstrap starting.
I20260923 22:11:39.093088 21785 tablet_bootstrap.cc:654] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:39.093146 21704 tablet_service.cc:1467] Processing CreateTablet for tablet 60e80766b0f547f49c613b833d78df38 (DEFAULT_TABLE table=test-table [id=776924b585f84030b7986c5cf0ce8eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 22:11:39.093410 21704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60e80766b0f547f49c613b833d78df38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:39.093402 21785 log.cc:824] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:39.093700 21784 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Bootstrap starting.
I20260923 22:11:39.094065 21785 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: No bootstrap required, opened a new log
I20260923 22:11:39.094129 21785 ts_tablet_manager.cc:1397] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:39.094237 21784 tablet_bootstrap.cc:654] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:39.094496 21784 log.cc:824] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:39.095112 21784 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: No bootstrap required, opened a new log
I20260923 22:11:39.095202 21784 ts_tablet_manager.cc:1397] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:39.096289 21788 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Bootstrap starting.
I20260923 22:11:39.096354 21785 raft_consensus.cc:348] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.096503 21785 raft_consensus.cc:374] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:39.096550 21785 raft_consensus.cc:729] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf7b92c2b2d44ddba03834d49cdca26, State: Initialized, Role: FOLLOWER
I20260923 22:11:39.096699 21785 consensus_queue.cc:260] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.096962 21785 ts_tablet_manager.cc:1428] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:39.097090 21508 heartbeater.cc:503] Master 127.1.161.190:36201 was elected leader, sending a full tablet report...
I20260923 22:11:39.097097 21788 tablet_bootstrap.cc:654] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:39.097203 21784 raft_consensus.cc:348] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.097360 21784 raft_consensus.cc:374] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:39.097381 21788 log.cc:824] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:39.097400 21784 raft_consensus.cc:729] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b436924fc6412b86109c37e1dd71a3, State: Initialized, Role: FOLLOWER
I20260923 22:11:39.097518 21784 consensus_queue.cc:260] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.097815 21784 ts_tablet_manager.cc:1428] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 22:11:39.097841 21639 heartbeater.cc:503] Master 127.1.161.190:36201 was elected leader, sending a full tablet report...
I20260923 22:11:39.098497 21788 tablet_bootstrap.cc:492] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: No bootstrap required, opened a new log
I20260923 22:11:39.098565 21788 ts_tablet_manager.cc:1397] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 22:11:39.100519 21788 raft_consensus.cc:348] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.100632 21788 raft_consensus.cc:374] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:39.100654 21788 raft_consensus.cc:729] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a3b84a2c7d4e58ad55c61f4d789d4a, State: Initialized, Role: FOLLOWER
I20260923 22:11:39.100761 21788 consensus_queue.cc:260] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.100941 21788 ts_tablet_manager.cc:1428] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 22:11:39.101034 21770 heartbeater.cc:503] Master 127.1.161.190:36201 was elected leader, sending a full tablet report...
W20260923 22:11:39.192174 21640 tablet.cc:2369] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:11:39.205183 21792 raft_consensus.cc:482] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 22:11:39.205271 21792 raft_consensus.cc:504] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.205554 21792 leader_election.cc:288] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6b436924fc6412b86109c37e1dd71a3 (127.1.161.130:43351), dcf7b92c2b2d44ddba03834d49cdca26 (127.1.161.129:39629)
I20260923 22:11:39.208276 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.205691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.130:43351 (local address 127.1.161.131:33555)
0923 22:11:39.205759 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.205771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.205818 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.205995 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.205996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.205998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:39.206068 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.206070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.206673 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.206674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.207488 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.207491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.208005 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.208006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.208093 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.208096 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.208163 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.208195 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 22:11:39.208292 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.205768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:33555 (local address 127.1.161.130:43351)
0923 22:11:39.205811 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.205813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.205820 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:11:39.205877 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.205878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.205893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.205942 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:39.206133 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.206584 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.207563 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.207872 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.207922 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.207928 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.208174 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.208214 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:11:39.208405 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.205849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.161.131:49101 (local address 127.1.161.129:39629)
0923 22:11:39.205888 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.205890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.205999 (+ 109us) server_negotiation.cc:407] Connection header received
0923 22:11:39.206031 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.206032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.206043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.206103 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 22:11:39.206286 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.206739 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.207606 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.207988 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.208091 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.208100 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.208284 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.208326 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:39.208570 21793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.205797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:39629 (local address 127.1.161.131:49101)
0923 22:11:39.205940 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.205948 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:11:39.205991 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:11:39.206128 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:11:39.206128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:11:39.206130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 22:11:39.206194 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.206196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.206813 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.206814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.207514 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:11:39.207515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.208104 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.208105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:11:39.208202 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.208206 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 22:11:39.208264 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 22:11:39.208300 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":19,"thread_start_us":83,"threads_started":1}
I20260923 22:11:39.208652 21593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e80766b0f547f49c613b833d78df38" candidate_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b436924fc6412b86109c37e1dd71a3" is_pre_election: true
I20260923 22:11:39.208806 21593 raft_consensus.cc:2454] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a3b84a2c7d4e58ad55c61f4d789d4a in term 0.
I20260923 22:11:39.208914 21462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e80766b0f547f49c613b833d78df38" candidate_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" is_pre_election: true
I20260923 22:11:39.209038 21658 leader_election.cc:302] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a3b84a2c7d4e58ad55c61f4d789d4a, c6b436924fc6412b86109c37e1dd71a3; no voters:
I20260923 22:11:39.209071 21462 raft_consensus.cc:2454] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a3b84a2c7d4e58ad55c61f4d789d4a in term 0.
I20260923 22:11:39.209192 21792 raft_consensus.cc:2781] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 22:11:39.209242 21792 raft_consensus.cc:482] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 22:11:39.209278 21792 raft_consensus.cc:3037] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:39.209915 21792 raft_consensus.cc:504] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.210055 21792 leader_election.cc:288] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [CANDIDATE]: Term 1 election: Requested vote from peers c6b436924fc6412b86109c37e1dd71a3 (127.1.161.130:43351), dcf7b92c2b2d44ddba03834d49cdca26 (127.1.161.129:39629)
I20260923 22:11:39.210184 21593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e80766b0f547f49c613b833d78df38" candidate_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b436924fc6412b86109c37e1dd71a3"
I20260923 22:11:39.210219 21462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60e80766b0f547f49c613b833d78df38" candidate_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf7b92c2b2d44ddba03834d49cdca26"
I20260923 22:11:39.210269 21593 raft_consensus.cc:3037] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:39.210275 21462 raft_consensus.cc:3037] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:39.210893 21593 raft_consensus.cc:2454] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a3b84a2c7d4e58ad55c61f4d789d4a in term 1.
I20260923 22:11:39.211025 21462 raft_consensus.cc:2454] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a3b84a2c7d4e58ad55c61f4d789d4a in term 1.
I20260923 22:11:39.211094 21658 leader_election.cc:302] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a3b84a2c7d4e58ad55c61f4d789d4a, c6b436924fc6412b86109c37e1dd71a3; no voters:
I20260923 22:11:39.211230 21792 raft_consensus.cc:2781] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:39.211395 21792 raft_consensus.cc:686] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 1 LEADER]: Becoming Leader. State: Replica: 87a3b84a2c7d4e58ad55c61f4d789d4a, State: Running, Role: LEADER
I20260923 22:11:39.211490 21792 consensus_queue.cc:237] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } }
I20260923 22:11:39.212222 21319 catalog_manager.cc:5495] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a3b84a2c7d4e58ad55c61f4d789d4a (127.1.161.131). New cstate: current_term: 1 leader_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87a3b84a2c7d4e58ad55c61f4d789d4a" member_type: VOTER last_known_addr { host: "127.1.161.131" port: 41257 } health_report { overall_health: HEALTHY } } }
I20260923 22:11:39.234642 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.232439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.1.161.131:41257)
0923 22:11:39.232488 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:39.232490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:39.232506 (+ 16us) server_negotiation.cc:407] Connection header received
0923 22:11:39.232543 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:39.232544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:39.232563 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:39.232624 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:11:39.232744 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.233140 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.233910 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:39.234099 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:39.234170 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:39.234316 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:11:39.234406 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:11:39.234494 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:39.234532 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:11:39.234548 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 22:11:39.236142 21462 raft_consensus.cc:1261] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26 [term 1 FOLLOWER]: Refusing update from remote peer 87a3b84a2c7d4e58ad55c61f4d789d4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:39.236157 21593 raft_consensus.cc:1261] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 1 FOLLOWER]: Refusing update from remote peer 87a3b84a2c7d4e58ad55c61f4d789d4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:11:39.236485 21792 consensus_queue.cc:1035] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6b436924fc6412b86109c37e1dd71a3" member_type: VOTER last_known_addr { host: "127.1.161.130" port: 43351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:39.236619 21794 consensus_queue.cc:1035] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcf7b92c2b2d44ddba03834d49cdca26" member_type: VOTER last_known_addr { host: "127.1.161.129" port: 39629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:11:39.239280 21798 mvcc.cc:205] Tried to move back new op lower bound from 7332665340869275648 to 7332665340770934784. Current Snapshot: MvccSnapshot[applied={T|T < 7332665340869275648}]
I20260923 22:11:39.239550 21800 mvcc.cc:205] Tried to move back new op lower bound from 7332665340869275648 to 7332665340770934784. Current Snapshot: MvccSnapshot[applied={T|T < 7332665340869275648}]
I20260923 22:11:39.240785 21319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:39.240875 21319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 22:11:39.241539 21319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21b85b48790e4285aa66e1bd4b496d71: Sending DeleteTablet for 3 replicas of tablet 60e80766b0f547f49c613b833d78df38
I20260923 22:11:39.241778 21573 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e80766b0f547f49c613b833d78df38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:39 UTC) from {username='slave'} at 127.0.0.1:58544
I20260923 22:11:39.241822 21442 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e80766b0f547f49c613b833d78df38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:39 UTC) from {username='slave'} at 127.0.0.1:54450
I20260923 22:11:39.242049 21704 tablet_service.cc:1514] Processing DeleteTablet for tablet 60e80766b0f547f49c613b833d78df38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 22:11:39 UTC) from {username='slave'} at 127.0.0.1:37344
I20260923 22:11:39.241990 21808 tablet_replica.cc:330] T 60e80766b0f547f49c613b833d78df38 P dcf7b92c2b2d44ddba03834d49cdca26: stopping tablet replica
I20260923 22:11:39.242187 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21380
I20260923 22:11:39.242692 21807 tablet_replica.cc:330] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: stopping tablet replica
I20260923 22:11:39.242790 21807 raft_consensus.cc:2229] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 22:11:39.242872 21807 raft_consensus.cc:2258] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:39.243247 21807 ts_tablet_manager.cc:1905] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:39.244458 21809 tablet_replica.cc:330] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: stopping tablet replica
I20260923 22:11:39.244542 21809 raft_consensus.cc:2229] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 1 LEADER]: Raft consensus shutting down.
I20260923 22:11:39.244618 21809 raft_consensus.cc:2258] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 22:11:39.245007 21807 ts_tablet_manager.cc:1918] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:39.245016 21809 ts_tablet_manager.cc:1905] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 22:11:39.245115 21807 log.cc:1196] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-1/wal/wals/60e80766b0f547f49c613b833d78df38
I20260923 22:11:39.245390 21807 ts_tablet_manager.cc:1939] T 60e80766b0f547f49c613b833d78df38 P c6b436924fc6412b86109c37e1dd71a3: Deleting consensus metadata
I20260923 22:11:39.245788 21305 catalog_manager.cc:4841] TS c6b436924fc6412b86109c37e1dd71a3 (127.1.161.130:43351): tablet 60e80766b0f547f49c613b833d78df38 (table test-table [id=776924b585f84030b7986c5cf0ce8eae]) successfully deleted
I20260923 22:11:39.246369 21809 ts_tablet_manager.cc:1918] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 22:11:39.246448 21809 log.cc:1196] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Deleting WAL directory at /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790201274647240-1670-0/minicluster-data/ts-2/wal/wals/60e80766b0f547f49c613b833d78df38
I20260923 22:11:39.246650 21809 ts_tablet_manager.cc:1939] T 60e80766b0f547f49c613b833d78df38 P 87a3b84a2c7d4e58ad55c61f4d789d4a: Deleting consensus metadata
I20260923 22:11:39.247021 21305 catalog_manager.cc:4841] TS 87a3b84a2c7d4e58ad55c61f4d789d4a (127.1.161.131:41257): tablet 60e80766b0f547f49c613b833d78df38 (table test-table [id=776924b585f84030b7986c5cf0ce8eae]) successfully deleted
W20260923 22:11:39.248354 21305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.161.129:39629 (error 108)
I20260923 22:11:39.248559 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21511
I20260923 22:11:39.249063 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:39.248645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.129:39629 (local address 127.0.0.1:54454)
0923 22:11:39.248992 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0923 22:11:39.249026 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.129:39629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331}
W20260923 22:11:39.251680 21305 catalog_manager.cc:4584] TS dcf7b92c2b2d44ddba03834d49cdca26 (127.1.161.129:39629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60e80766b0f547f49c613b833d78df38: Network error: Client connection negotiation failed: client connection to 127.1.161.129:39629: connect: Connection refused (error 111)
I20260923 22:11:39.253055 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21642
I20260923 22:11:39.257588 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21289
2026-09-23T22:11:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (717 ms)
[----------] 6 tests from EncryptionPolicyTest (7832 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:39 dist-test-slave-npvd krb5kdc[21813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:39 dist-test-slave-npvd krb5kdc[21813](info): set up 2 sockets
Sep 23 22:11:39 dist-test-slave-npvd krb5kdc[21813](info): commencing operation
krb5kdc: starting...
W20260923 22:11:40.198941 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:40.299798 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:40.375481 21826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:40.375663 21826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:40.375692 21826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:40.376750 21826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:40.376828 21826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:40.376852 21826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:40.376871 21826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:40.376899 21826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:40.378392 21826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:40.378644 21826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:40.378854 21826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:40.383684 21834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:40.383762 21831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:40.383684 21832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:40.383718 21826 server_base.cc:1034] running on GCE node
I20260923 22:11:40.384119 21826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:40.384320 21826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:40.385465 21826 hybrid_clock.cc:648] HybridClock initialized: now 1790201500385455 us; error 25 us; skew 500 ppm
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:40.387727 21826 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:40.388121 21826 webserver.cc:460] Webserver started at http://127.0.0.1:37769/ using document root <none> and password file <none>
I20260923 22:11:40.388335 21826 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:40.388374 21826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:40.388455 21826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:40.389374 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "a1471f3e574c418abd3774ad5bd04540"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-npvd"
server_key: "88b2c7dec5befed30dae38a1c36542b0"
server_key_iv: "c40aa03fe1c38d741dfb011b2592f254"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.389624 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "a1471f3e574c418abd3774ad5bd04540"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-npvd"
server_key: "88b2c7dec5befed30dae38a1c36542b0"
server_key_iv: "c40aa03fe1c38d741dfb011b2592f254"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.390664 21826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:40.391296 21841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:40.391458 21826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:40.391547 21826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "a1471f3e574c418abd3774ad5bd04540"
format_stamp: "Formatted at 2026-09-23 22:11:40 on dist-test-slave-npvd"
server_key: "88b2c7dec5befed30dae38a1c36542b0"
server_key_iv: "c40aa03fe1c38d741dfb011b2592f254"
server_key_version: "encryptionkey@0"
I20260923 22:11:40.391610 21826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:40.423552 21826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:40.425815 21826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:40.425963 21826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:40.429922 21826 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39279
I20260923 22:11:40.429949 21893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39279 every 8 connection(s)
I20260923 22:11:40.430274 21826 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:40.431190 21894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:40.433130 21894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540: Bootstrap starting.
I20260923 22:11:40.433565 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21826
I20260923 22:11:40.433686 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:40.433758 21894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:40.433907 1670 external_mini_cluster.cc:1428] Setting key a298edf4ef94d4f92784128be94f689a
I20260923 22:11:40.434057 21894 log.cc:824] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:40.434767 21894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540: No bootstrap required, opened a new log
I20260923 22:11:40.436709 21894 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } }
I20260923 22:11:40.436832 21894 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:40.436853 21894 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1471f3e574c418abd3774ad5bd04540, State: Initialized, Role: FOLLOWER
I20260923 22:11:40.436997 21894 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } }
I20260923 22:11:40.437067 21894 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:40.437115 21894 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:40.437161 21894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:40.437674 21894 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } }
I20260923 22:11:40.437788 21894 leader_election.cc:302] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1471f3e574c418abd3774ad5bd04540; no voters:
I20260923 22:11:40.437988 21894 leader_election.cc:288] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:40.438010 21899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:40.438136 21899 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [term 1 LEADER]: Becoming Leader. State: Replica: a1471f3e574c418abd3774ad5bd04540, State: Running, Role: LEADER
I20260923 22:11:40.438266 21894 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:40.438293 21899 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } }
I20260923 22:11:40.438653 21900 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1471f3e574c418abd3774ad5bd04540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } } }
I20260923 22:11:40.438637 21899 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1471f3e574c418abd3774ad5bd04540. Latest consensus state: current_term: 1 leader_uuid: "a1471f3e574c418abd3774ad5bd04540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1471f3e574c418abd3774ad5bd04540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39279 } } }
I20260923 22:11:40.438903 21899 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:40.438877 21900 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:40.439127 21908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:40.439610 21908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:40.441164 21908 catalog_manager.cc:1347] Generated new cluster ID: 9d8f4b6af0564bc4ba6e3029fe6979fd
I20260923 22:11:40.441211 21908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:40.445149 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:40.434365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59117 (local address 127.0.0.1:39279)
0923 22:11:40.434565 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:40.434574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:40.434603 (+ 29us) server_negotiation.cc:407] Connection header received
0923 22:11:40.435085 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:40.435091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:40.435134 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:40.435342 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:40.436004 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.436891 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.437721 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.438375 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.438767 (+ 392us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:40.441995 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:40.442000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:40.442000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:40.442011 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:40.443653 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:40.444019 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:40.444021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:40.444022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:40.444107 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:40.444332 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:40.444333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:40.444334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:40.444536 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:40.444699 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:40.444797 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:11:40.444884 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260923 22:11:40.446059 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:40.456272 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:40.447273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59727 (local address 127.0.0.1:39279)
0923 22:11:40.447308 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:40.447309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:40.451307 (+ 3998us) server_negotiation.cc:407] Connection header received
0923 22:11:40.451367 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:40.451368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:40.451376 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:40.451412 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:40.451854 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.452262 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.452844 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:40.453041 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:40.453106 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:40.454060 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:40.454061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:40.454061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:40.454067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:40.455274 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:40.455582 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:40.455583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:40.455583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:40.455627 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:40.455827 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:40.455827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:40.455828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:40.455920 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:40.456014 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:40.456121 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 22:11:40.456173 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:40.457376 21908 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:40.458020 21908 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:40.465363 21908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1471f3e574c418abd3774ad5bd04540: Generated new TSK 0
I20260923 22:11:40.465495 21908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:40.471709 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21826
W20260923 22:11:40.475522 21811 connection.cc:535] client connection to 127.0.0.1:39279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39279): Connection reset by peer (error 104)
2026-09-23T22:11:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21925](info): set up 2 sockets
Sep 23 22:11:40 dist-test-slave-npvd krb5kdc[21925](info): commencing operation
krb5kdc: starting...
W20260923 22:11:41.414736 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:41 dist-test-slave-npvd krb5kdc[21925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:41.515559 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:41.587479 21938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:41.587635 21938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:41.587656 21938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:41.588726 21938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:41.588764 21938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:41.588778 21938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:41.588791 21938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:41.588804 21938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:41.590106 21938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:41.590281 21938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:41.590415 21938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:41.594668 21944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:41.594681 21946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:41.594803 21938 server_base.cc:1034] running on GCE node
W20260923 22:11:41.594728 21943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:41.595075 21938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:41.595371 21938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:41.596525 21938 hybrid_clock.cc:648] HybridClock initialized: now 1790201501596511 us; error 41 us; skew 500 ppm
Sep 23 22:11:41 dist-test-slave-npvd krb5kdc[21925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:41.604377 21938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:41.604832 21938 webserver.cc:460] Webserver started at http://127.0.0.1:33049/ using document root <none> and password file <none>
I20260923 22:11:41.605036 21938 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:41.605077 21938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:41.605157 21938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:41.606063 21938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "bb9cba695567469c98862ffca78c3312"
format_stamp: "Formatted at 2026-09-23 22:11:41 on dist-test-slave-npvd"
server_key: "c2f035195c8e7cb8bb8d0ebbaa9c0992"
server_key_iv: "69b0ce655a9ceb204b6441352707537f"
server_key_version: "encryptionkey@0"
I20260923 22:11:41.606351 21938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "bb9cba695567469c98862ffca78c3312"
format_stamp: "Formatted at 2026-09-23 22:11:41 on dist-test-slave-npvd"
server_key: "c2f035195c8e7cb8bb8d0ebbaa9c0992"
server_key_iv: "69b0ce655a9ceb204b6441352707537f"
server_key_version: "encryptionkey@0"
I20260923 22:11:41.607561 21938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:41.608170 21953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:41.608368 21938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:41.608438 21938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "bb9cba695567469c98862ffca78c3312"
format_stamp: "Formatted at 2026-09-23 22:11:41 on dist-test-slave-npvd"
server_key: "c2f035195c8e7cb8bb8d0ebbaa9c0992"
server_key_iv: "69b0ce655a9ceb204b6441352707537f"
server_key_version: "encryptionkey@0"
I20260923 22:11:41.608532 21938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:41.620038 21938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:41.621618 21938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:41.621762 21938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:41.625550 21938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40831
I20260923 22:11:41.625576 22005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40831 every 8 connection(s)
I20260923 22:11:41.625873 21938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:41.626667 22006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:41.628638 22006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312: Bootstrap starting.
I20260923 22:11:41.629000 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 21938
I20260923 22:11:41.629135 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:41.629331 22006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:41.629354 1670 external_mini_cluster.cc:1428] Setting key e8da1f3376a4569291a7249180b623b8
I20260923 22:11:41.629632 22006 log.cc:824] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:41.630259 22006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312: No bootstrap required, opened a new log
I20260923 22:11:41.631824 22006 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260923 22:11:41.631980 22006 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:41.632022 22006 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9cba695567469c98862ffca78c3312, State: Initialized, Role: FOLLOWER
I20260923 22:11:41.632133 22006 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260923 22:11:41.632206 22006 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:41.632246 22006 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:41.632288 22006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:41.632788 22006 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260923 22:11:41.632894 22006 leader_election.cc:302] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb9cba695567469c98862ffca78c3312; no voters:
I20260923 22:11:41.633050 22006 leader_election.cc:288] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:41.633080 22011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:41.633198 22011 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [term 1 LEADER]: Becoming Leader. State: Replica: bb9cba695567469c98862ffca78c3312, State: Running, Role: LEADER
I20260923 22:11:41.633276 22006 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:41.633303 22011 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260923 22:11:41.633679 22013 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb9cba695567469c98862ffca78c3312. Latest consensus state: current_term: 1 leader_uuid: "bb9cba695567469c98862ffca78c3312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } } }
I20260923 22:11:41.633798 22013 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:41.633849 22012 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb9cba695567469c98862ffca78c3312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9cba695567469c98862ffca78c3312" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } } }
I20260923 22:11:41.633981 22012 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:41.634292 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:41.629773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34457 (local address 127.0.0.1:40831)
0923 22:11:41.629941 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:41.629949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:41.629980 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:11:41.630359 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:41.630366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:41.630411 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:41.630541 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:41.630730 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:41.631533 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:41.632396 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:41.632770 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:41.632878 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:41.633493 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:41.633496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:41.633496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:41.633686 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:41.633721 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:41.633734 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:41.633815 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:41.634049 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 22:11:41.634076 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260923 22:11:41.634127 22020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:41.635182 22020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:41.635202 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:41.636643 22020 catalog_manager.cc:1347] Generated new cluster ID: 4b69ccfb9c59433cb5d65b9a6641b2d5
I20260923 22:11:41.636726 22020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:41.640020 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:41.636471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53477 (local address 127.0.0.1:40831)
0923 22:11:41.636502 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:41.636504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:41.636786 (+ 282us) server_negotiation.cc:407] Connection header received
0923 22:11:41.636886 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:41.636887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:41.636896 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:41.636968 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:41.637132 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:41.637788 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:41.638775 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:41.639086 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:41.639140 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:41.639798 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:41.639798 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:41.639799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:41.639802 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:41.639814 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:41.639817 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:41.639862 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:41.639946 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:11:41.639954 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:41.644173 22020 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:41.644614 22020 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:41.646813 22020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb9cba695567469c98862ffca78c3312: Generated new TSK 0
I20260923 22:11:41.646961 22020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:41.653162 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 21938
W20260923 22:11:41.656749 21923 connection.cc:535] client connection to 127.0.0.1:40831 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40831): Connection reset by peer (error 104)
2026-09-23T22:11:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:41 dist-test-slave-npvd krb5kdc[22037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:41 dist-test-slave-npvd krb5kdc[22037](info): set up 2 sockets
Sep 23 22:11:41 dist-test-slave-npvd krb5kdc[22037](info): commencing operation
krb5kdc: starting...
W20260923 22:11:42.586568 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:42 dist-test-slave-npvd krb5kdc[22037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:42.685305 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:42.756122 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:42.756289 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:42.756309 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:42.757335 22050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:42.757375 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:42.757388 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:42.757433 22050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:42.757458 22050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:42.759147 22050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:42.759444 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:42.759689 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:42.763895 22056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:42.763919 22058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:42.763962 22050 server_base.cc:1034] running on GCE node
W20260923 22:11:42.763902 22055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:42.764264 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:42.764468 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:42.765595 22050 hybrid_clock.cc:648] HybridClock initialized: now 1790201502765583 us; error 27 us; skew 500 ppm
Sep 23 22:11:42 dist-test-slave-npvd krb5kdc[22037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:42.767715 22050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:42.768085 22050 webserver.cc:460] Webserver started at http://127.0.0.1:46689/ using document root <none> and password file <none>
I20260923 22:11:42.768316 22050 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:42.768360 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:42.768496 22050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:42.769470 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "912eebd2b2004fe6a6e33ea37f0a8aea"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-npvd"
server_key: "b8696dde14fd876bbe6413a083ffeeda"
server_key_iv: "292ba35658d54e5f50f2504090b361f9"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.769804 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "912eebd2b2004fe6a6e33ea37f0a8aea"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-npvd"
server_key: "b8696dde14fd876bbe6413a083ffeeda"
server_key_iv: "292ba35658d54e5f50f2504090b361f9"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.770970 22050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:42.771646 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:42.771798 22050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:42.771879 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "912eebd2b2004fe6a6e33ea37f0a8aea"
format_stamp: "Formatted at 2026-09-23 22:11:42 on dist-test-slave-npvd"
server_key: "b8696dde14fd876bbe6413a083ffeeda"
server_key_iv: "292ba35658d54e5f50f2504090b361f9"
server_key_version: "encryptionkey@0"
I20260923 22:11:42.771963 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:42.774660 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:42.776302 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:42.776437 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:42.780236 22050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37771
I20260923 22:11:42.780262 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37771 every 8 connection(s)
I20260923 22:11:42.780521 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:42.781337 22118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:42.783125 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea: Bootstrap starting.
I20260923 22:11:42.783617 22118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:42.783815 22118 log.cc:824] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:42.784332 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea: No bootstrap required, opened a new log
I20260923 22:11:42.785619 22118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } }
I20260923 22:11:42.785725 22118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:42.785744 22118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912eebd2b2004fe6a6e33ea37f0a8aea, State: Initialized, Role: FOLLOWER
I20260923 22:11:42.785825 22118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } }
I20260923 22:11:42.785867 22118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:42.785885 22118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:42.785907 22118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:42.786289 22118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } }
I20260923 22:11:42.786360 22118 leader_election.cc:302] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912eebd2b2004fe6a6e33ea37f0a8aea; no voters:
I20260923 22:11:42.786464 22118 leader_election.cc:288] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:42.786528 22121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:42.786657 22121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [term 1 LEADER]: Becoming Leader. State: Replica: 912eebd2b2004fe6a6e33ea37f0a8aea, State: Running, Role: LEADER
I20260923 22:11:42.786706 22118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:42.786768 22121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } }
I20260923 22:11:42.787096 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912eebd2b2004fe6a6e33ea37f0a8aea. Latest consensus state: current_term: 1 leader_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } } }
I20260923 22:11:42.787231 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:42.787097 22122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912eebd2b2004fe6a6e33ea37f0a8aea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37771 } } }
I20260923 22:11:42.787423 22125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:42.787457 22122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:42.787932 22125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:42.788699 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22050
I20260923 22:11:42.788825 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:42.789022 1670 external_mini_cluster.cc:1428] Setting key 924347f43ed7ad41944e398aa9d5c4f0
I20260923 22:11:42.789458 22125 catalog_manager.cc:1347] Generated new cluster ID: 730ecb6b668941ce80eaf6cf022179f7
I20260923 22:11:42.789517 22125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:42.790973 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:42.789608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55575 (local address 127.0.0.1:37771)
0923 22:11:42.789734 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:42.789741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:42.789766 (+ 25us) server_negotiation.cc:407] Connection header received
0923 22:11:42.789853 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:42.789858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:42.789884 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:42.789989 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:42.790588 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:42.790591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:42.790591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:42.790594 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:42.790629 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:42.790640 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:42.790694 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:42.790790 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 22:11:42.790823 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
I20260923 22:11:42.791853 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:42.794899 22125 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:42.795032 22125 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:42.803329 22125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912eebd2b2004fe6a6e33ea37f0a8aea: Generated new TSK 0
I20260923 22:11:42.803498 22125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:42.804028 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:42.794822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60593 (local address 127.0.0.1:37771)
0923 22:11:42.794852 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:42.794854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:42.803243 (+ 8389us) server_negotiation.cc:407] Connection header received
0923 22:11:42.803295 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:42.803295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:42.803308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:42.803360 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:42.803847 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:42.803848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:42.803848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:42.803851 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:42.803861 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:42.803864 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:42.803898 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:42.803948 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:11:42.803957 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:11:42.805177 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22050
2026-09-23T22:11:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:42 dist-test-slave-npvd krb5kdc[22149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:42 dist-test-slave-npvd krb5kdc[22149](info): set up 2 sockets
Sep 23 22:11:42 dist-test-slave-npvd krb5kdc[22149](info): commencing operation
krb5kdc: starting...
W20260923 22:11:43.730993 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.909s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:43 dist-test-slave-npvd krb5kdc[22149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:43.831565 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:43.903736 22162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:43.903891 22162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:43.903910 22162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:43.904894 22162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:43.904932 22162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:43.904944 22162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:43.904989 22162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:43.905014 22162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:43.906226 22162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:43.906436 22162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:43.906620 22162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:43.911096 22167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:43.911062 22168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:43.911201 22170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:43.911062 22162 server_base.cc:1034] running on GCE node
I20260923 22:11:43.911511 22162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:43.911777 22162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:43.912909 22162 hybrid_clock.cc:648] HybridClock initialized: now 1790201503912891 us; error 34 us; skew 500 ppm
Sep 23 22:11:43 dist-test-slave-npvd krb5kdc[22149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:43.915047 22162 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:43.915503 22162 webserver.cc:460] Webserver started at http://127.0.0.1:41765/ using document root <none> and password file <none>
I20260923 22:11:43.915700 22162 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:43.915760 22162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:43.915877 22162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:43.916749 22162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-npvd"
server_key: "baa22b580a7d163ef14e788f05f8a18c"
server_key_iv: "55194c71f97c60734d525ad13885eb79"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.917073 22162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-npvd"
server_key: "baa22b580a7d163ef14e788f05f8a18c"
server_key_iv: "55194c71f97c60734d525ad13885eb79"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.918202 22162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:43.918864 22177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:43.919041 22162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:43.919140 22162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0"
format_stamp: "Formatted at 2026-09-23 22:11:43 on dist-test-slave-npvd"
server_key: "baa22b580a7d163ef14e788f05f8a18c"
server_key_iv: "55194c71f97c60734d525ad13885eb79"
server_key_version: "encryptionkey@0"
I20260923 22:11:43.919226 22162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:43.931836 22162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:43.933737 22162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:43.933873 22162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:43.937408 22162 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34181
I20260923 22:11:43.937438 22229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34181 every 8 connection(s)
I20260923 22:11:43.937714 22162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:43.938544 22230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:43.940579 22230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0: Bootstrap starting.
I20260923 22:11:43.940989 22230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:43.941186 22230 log.cc:824] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:43.941701 22230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0: No bootstrap required, opened a new log
I20260923 22:11:43.942962 22230 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } }
I20260923 22:11:43.943068 22230 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:43.943089 22230 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d59205c45f4d52bdc02bf14cbf1bf0, State: Initialized, Role: FOLLOWER
I20260923 22:11:43.943216 22230 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } }
I20260923 22:11:43.943282 22230 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:43.943302 22230 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:43.943326 22230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:43.943713 22230 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } }
I20260923 22:11:43.943782 22230 leader_election.cc:302] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7d59205c45f4d52bdc02bf14cbf1bf0; no voters:
I20260923 22:11:43.943881 22230 leader_election.cc:288] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:43.943938 22233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:43.944092 22233 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [term 1 LEADER]: Becoming Leader. State: Replica: d7d59205c45f4d52bdc02bf14cbf1bf0, State: Running, Role: LEADER
I20260923 22:11:43.944101 22230 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:43.944240 22233 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } }
I20260923 22:11:43.944603 22235 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7d59205c45f4d52bdc02bf14cbf1bf0. Latest consensus state: current_term: 1 leader_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } } }
I20260923 22:11:43.944674 22235 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:43.944694 22234 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7d59205c45f4d52bdc02bf14cbf1bf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34181 } } }
I20260923 22:11:43.944752 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22162
I20260923 22:11:43.944792 22234 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:43.944850 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:43.945027 1670 external_mini_cluster.cc:1428] Setting key 9088017220573c14db6452a52fd28ba6
I20260923 22:11:43.945498 22239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:43.945971 22239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:43.948173 22239 catalog_manager.cc:1347] Generated new cluster ID: 6667405369e74842905c02aceca6f610
I20260923 22:11:43.948275 22239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:43 dist-test-slave-npvd krb5kdc[22149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:43.954417 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:43.945519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36625 (local address 127.0.0.1:34181)
0923 22:11:43.945838 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:43.945847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:43.945879 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:11:43.945986 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:43.945992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:43.946045 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:43.946188 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:43.946877 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.948056 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.948792 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.949209 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.949327 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:43.951317 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:43.951321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:43.951321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:43.951331 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:43.953112 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.953430 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.953433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.953433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.953518 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.953724 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.953725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.953726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.953931 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:43.954088 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:43.954146 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:11:43.954264 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":70,"threads_started":1}
I20260923 22:11:43.955147 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:43.958817 22239 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:43.959486 22239 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:43.964926 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:43.958648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59303 (local address 127.0.0.1:34181)
0923 22:11:43.958698 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:43.958700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:43.959697 (+ 997us) server_negotiation.cc:407] Connection header received
0923 22:11:43.959757 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:43.959758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:43.959767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:43.959804 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:43.960247 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.960665 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.961281 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:43.961457 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:43.961523 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:43.962450 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:43.962452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:43.962452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:43.962457 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:43.963929 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.964233 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.964234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.964235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.964310 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:43.964498 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:43.964499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:43.964499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:43.964605 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:43.964700 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:43.964783 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:11:43.964846 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 22:11:43.971573 22239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7d59205c45f4d52bdc02bf14cbf1bf0: Generated new TSK 0
I20260923 22:11:43.971709 22239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:43.975023 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22162
2026-09-23T22:11:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:44 dist-test-slave-npvd krb5kdc[22261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:44 dist-test-slave-npvd krb5kdc[22261](info): set up 2 sockets
Sep 23 22:11:44 dist-test-slave-npvd krb5kdc[22261](info): commencing operation
krb5kdc: starting...
W20260923 22:11:44.914608 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:44 dist-test-slave-npvd krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:45.015513 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:45.084802 22274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:45.084993 22274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:45.085026 22274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:45.086066 22274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:45.086125 22274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:45.086146 22274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:45.086165 22274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:45.086241 22274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:45.087528 22274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:45.087777 22274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:45.087965 22274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:45.092401 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:45.092430 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:45.092440 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:45.092521 22274 server_base.cc:1034] running on GCE node
I20260923 22:11:45.092748 22274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:45.093017 22274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:45.094183 22274 hybrid_clock.cc:648] HybridClock initialized: now 1790201505094165 us; error 33 us; skew 500 ppm
Sep 23 22:11:45 dist-test-slave-npvd krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:45.096594 22274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:45.097028 22274 webserver.cc:460] Webserver started at http://127.0.0.1:43961/ using document root <none> and password file <none>
I20260923 22:11:45.097234 22274 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:45.097289 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:45.097438 22274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:45.098311 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "dcc9adead11240c090449a2a2d11acf3"
format_stamp: "Formatted at 2026-09-23 22:11:45 on dist-test-slave-npvd"
server_key: "3fa38c426e0a7e642765ab8407207fcc"
server_key_iv: "ed189ededfbd9a6afffbdd731454d4b4"
server_key_version: "encryptionkey@0"
I20260923 22:11:45.098670 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "dcc9adead11240c090449a2a2d11acf3"
format_stamp: "Formatted at 2026-09-23 22:11:45 on dist-test-slave-npvd"
server_key: "3fa38c426e0a7e642765ab8407207fcc"
server_key_iv: "ed189ededfbd9a6afffbdd731454d4b4"
server_key_version: "encryptionkey@0"
I20260923 22:11:45.100034 22274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:45.100688 22289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:45.100857 22274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:45.100942 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "dcc9adead11240c090449a2a2d11acf3"
format_stamp: "Formatted at 2026-09-23 22:11:45 on dist-test-slave-npvd"
server_key: "3fa38c426e0a7e642765ab8407207fcc"
server_key_iv: "ed189ededfbd9a6afffbdd731454d4b4"
server_key_version: "encryptionkey@0"
I20260923 22:11:45.101003 22274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:45.111456 22274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:45.112907 22274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:45.113068 22274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:45.116397 22274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42597
I20260923 22:11:45.116425 22341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42597 every 8 connection(s)
I20260923 22:11:45.116703 22274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:45.117502 22342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:45.118875 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22274
I20260923 22:11:45.119007 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:45.119252 1670 external_mini_cluster.cc:1428] Setting key 1589a6684420544e0d4f81ae2d0a55e6
I20260923 22:11:45.119786 22342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3: Bootstrap starting.
I20260923 22:11:45.120297 22342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:45.120558 22342 log.cc:824] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:45.121151 22342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3: No bootstrap required, opened a new log
I20260923 22:11:45.122576 22342 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260923 22:11:45.122704 22342 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:45.122767 22342 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc9adead11240c090449a2a2d11acf3, State: Initialized, Role: FOLLOWER
I20260923 22:11:45.122905 22342 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260923 22:11:45.122985 22342 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:45.123016 22342 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:45.123062 22342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:45.123577 22342 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260923 22:11:45.123709 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:45.119710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49285 (local address 127.0.0.1:42597)
0923 22:11:45.119892 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:45.119900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:45.119938 (+ 38us) server_negotiation.cc:407] Connection header received
0923 22:11:45.120274 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:45.120278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:45.120313 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:45.120476 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:45.120661 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:45.121432 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:45.122107 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:45.122444 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:45.122556 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:45.123124 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:45.123128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:45.123128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:45.123308 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:45.123338 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:45.123347 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:45.123409 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:45.123486 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:11:45.123539 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260923 22:11:45.123689 22342 leader_election.cc:302] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc9adead11240c090449a2a2d11acf3; no voters:
I20260923 22:11:45.123873 22342 leader_election.cc:288] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:45.123921 22347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:45.124043 22347 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [term 1 LEADER]: Becoming Leader. State: Replica: dcc9adead11240c090449a2a2d11acf3, State: Running, Role: LEADER
I20260923 22:11:45.124080 22342 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:45.124189 22347 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } }
I20260923 22:11:45.124454 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:45.124529 22349 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc9adead11240c090449a2a2d11acf3. Latest consensus state: current_term: 1 leader_uuid: "dcc9adead11240c090449a2a2d11acf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } } }
I20260923 22:11:45.124593 22349 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:45.124709 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc9adead11240c090449a2a2d11acf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc9adead11240c090449a2a2d11acf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42597 } } }
I20260923 22:11:45.124754 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:45.124985 22355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:45.125382 22355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:45.126926 22355 catalog_manager.cc:1347] Generated new cluster ID: dc44244c35d243b983e16c148cc6da99
I20260923 22:11:45.126972 22355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:45.128484 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:45.125963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47535 (local address 127.0.0.1:42597)
0923 22:11:45.125993 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:45.125995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:45.126093 (+ 98us) server_negotiation.cc:407] Connection header received
0923 22:11:45.126159 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:45.126160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:45.126167 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:45.126213 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:45.126301 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:45.126855 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:45.127564 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:45.127748 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:45.127811 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:45.128299 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:45.128300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:45.128301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:45.128304 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:45.128314 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:45.128317 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:45.128357 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:45.128400 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 22:11:45.128422 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:11:45.139309 22355 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:45.139763 22355 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:45.146579 22355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc9adead11240c090449a2a2d11acf3: Generated new TSK 0
I20260923 22:11:45.146731 22355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:45.148934 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22274
2026-09-23T22:11:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:45 dist-test-slave-npvd krb5kdc[22373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:45 dist-test-slave-npvd krb5kdc[22373](info): set up 2 sockets
Sep 23 22:11:45 dist-test-slave-npvd krb5kdc[22373](info): commencing operation
krb5kdc: starting...
W20260923 22:11:46.086563 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:46 dist-test-slave-npvd krb5kdc[22373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:46.186812 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:46.257359 22386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:46.257520 22386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:46.257539 22386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:46.258600 22386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:46.258641 22386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:46.258653 22386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:46.258665 22386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:46.258733 22386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:46.260205 22386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:46.260418 22386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:46.260610 22386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:46.264782 22394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:46.264803 22386 server_base.cc:1034] running on GCE node
W20260923 22:11:46.264783 22391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:46.264810 22392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:46.265184 22386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:46.265425 22386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:46.266569 22386 hybrid_clock.cc:648] HybridClock initialized: now 1790201506266564 us; error 28 us; skew 500 ppm
Sep 23 22:11:46 dist-test-slave-npvd krb5kdc[22373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201506, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:46.268818 22386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:46.269265 22386 webserver.cc:460] Webserver started at http://127.0.0.1:35487/ using document root <none> and password file <none>
I20260923 22:11:46.269484 22386 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:46.269550 22386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:46.269685 22386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:46.270563 22386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "10f5b15d64a0441eba02bf24e9666b22"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-npvd"
server_key: "d7da5def34082b80c70751adcb862a58"
server_key_iv: "a2000b315088ca276da7961ca1d2faff"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.270862 22386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "10f5b15d64a0441eba02bf24e9666b22"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-npvd"
server_key: "d7da5def34082b80c70751adcb862a58"
server_key_iv: "a2000b315088ca276da7961ca1d2faff"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.272053 22386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:46.272703 22401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:46.272888 22386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:11:46.272993 22386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "10f5b15d64a0441eba02bf24e9666b22"
format_stamp: "Formatted at 2026-09-23 22:11:46 on dist-test-slave-npvd"
server_key: "d7da5def34082b80c70751adcb862a58"
server_key_iv: "a2000b315088ca276da7961ca1d2faff"
server_key_version: "encryptionkey@0"
I20260923 22:11:46.273067 22386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:46.276171 22386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:46.278187 22386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:46.278321 22386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:46.281913 22386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37635
I20260923 22:11:46.281944 22453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37635 every 8 connection(s)
I20260923 22:11:46.282229 22386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:46.283044 22454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:46.284925 22454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22: Bootstrap starting.
I20260923 22:11:46.285372 22454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:46.285575 22454 log.cc:824] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:46.286157 22454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22: No bootstrap required, opened a new log
I20260923 22:11:46.287693 22454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260923 22:11:46.287818 22454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:46.287839 22454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f5b15d64a0441eba02bf24e9666b22, State: Initialized, Role: FOLLOWER
I20260923 22:11:46.287930 22454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260923 22:11:46.287976 22454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:46.287995 22454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:46.288024 22454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:46.288466 22454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260923 22:11:46.288542 22454 leader_election.cc:302] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10f5b15d64a0441eba02bf24e9666b22; no voters:
I20260923 22:11:46.288656 22454 leader_election.cc:288] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:46.288754 22457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:46.288906 22457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [term 1 LEADER]: Becoming Leader. State: Replica: 10f5b15d64a0441eba02bf24e9666b22, State: Running, Role: LEADER
I20260923 22:11:46.288946 22454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:46.289044 22457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } }
I20260923 22:11:46.289389 22459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10f5b15d64a0441eba02bf24e9666b22. Latest consensus state: current_term: 1 leader_uuid: "10f5b15d64a0441eba02bf24e9666b22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260923 22:11:46.289505 22458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10f5b15d64a0441eba02bf24e9666b22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10f5b15d64a0441eba02bf24e9666b22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37635 } } }
I20260923 22:11:46.289541 22459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:46.289630 22458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:46.290021 22467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:46.290295 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22386
I20260923 22:11:46.290416 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:46.290501 22467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:46.290617 1670 external_mini_cluster.cc:1428] Setting key fdf077c51e2201aaed2d7b87e1ac0072
I20260923 22:11:46.291965 22467 catalog_manager.cc:1347] Generated new cluster ID: 549b675dcde84850be9d1c7e19aace58
I20260923 22:11:46.292025 22467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:46.295730 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:46.291007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45745 (local address 127.0.0.1:37635)
0923 22:11:46.291254 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:46.291261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:46.291289 (+ 28us) server_negotiation.cc:407] Connection header received
0923 22:11:46.291390 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:46.291396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:46.291424 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:46.291551 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:46.295371 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:46.295375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:46.295375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:46.295379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:46.295417 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:46.295430 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:46.295496 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:46.295554 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 22:11:46.295589 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260923 22:11:46.296783 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:46.299242 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:46.298212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49433 (local address 127.0.0.1:37635)
0923 22:11:46.298241 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:46.298242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:46.298345 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:11:46.298426 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:46.298427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:46.298431 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:46.298473 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:46.299043 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:46.299043 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:46.299044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:46.299046 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:46.299057 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:46.299060 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:46.299102 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:46.299179 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 22:11:46.299190 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:11:46.304240 22467 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:46.304359 22467 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:46.309589 22467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10f5b15d64a0441eba02bf24e9666b22: Generated new TSK 0
I20260923 22:11:46.309733 22467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:46.314286 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22386
2026-09-23T22:11:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1165 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:46 dist-test-slave-npvd krb5kdc[22485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:46 dist-test-slave-npvd krb5kdc[22485](info): set up 2 sockets
Sep 23 22:11:46 dist-test-slave-npvd krb5kdc[22485](info): commencing operation
krb5kdc: starting...
W20260923 22:11:47.250998 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:47.351841 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:47.423058 22498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:47.423290 22498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:47.423326 22498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:47.424291 22498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:47.424352 22498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:47.424372 22498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:47.424391 22498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:47.424441 22498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:47.425689 22498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:47.425930 22498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:47.426124 22498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:11:47.430532 22498 server_base.cc:1034] running on GCE node
W20260923 22:11:47.430492 22504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:47.430506 22503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:47.430495 22506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:47.430831 22498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:47.431113 22498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:47.432258 22498 hybrid_clock.cc:648] HybridClock initialized: now 1790201507432250 us; error 24 us; skew 500 ppm
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:47.439615 22498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:47.440021 22498 webserver.cc:460] Webserver started at http://127.0.0.1:44571/ using document root <none> and password file <none>
I20260923 22:11:47.440228 22498 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:47.440286 22498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:47.440411 22498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:47.441259 22498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "a052243e97bf4ecab149ff9c00d270e7"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-npvd"
server_key: "33493dd9a87655f2bdf3d2daf587b89f"
server_key_iv: "d14b81ccd777fbc7faa3d9e705df73a0"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.441567 22498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "a052243e97bf4ecab149ff9c00d270e7"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-npvd"
server_key: "33493dd9a87655f2bdf3d2daf587b89f"
server_key_iv: "d14b81ccd777fbc7faa3d9e705df73a0"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.442826 22498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:11:47.443532 22513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:47.443783 22498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:47.443866 22498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "a052243e97bf4ecab149ff9c00d270e7"
format_stamp: "Formatted at 2026-09-23 22:11:47 on dist-test-slave-npvd"
server_key: "33493dd9a87655f2bdf3d2daf587b89f"
server_key_iv: "d14b81ccd777fbc7faa3d9e705df73a0"
server_key_version: "encryptionkey@0"
I20260923 22:11:47.443940 22498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:47.460741 22498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:47.462531 22498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:47.462670 22498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:47.465773 22498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44517
I20260923 22:11:47.465804 22565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44517 every 8 connection(s)
I20260923 22:11:47.466084 22498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:47.466843 22566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:47.468540 22566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7: Bootstrap starting.
I20260923 22:11:47.469022 22566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:47.469239 22566 log.cc:824] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:47.469835 22566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7: No bootstrap required, opened a new log
I20260923 22:11:47.471107 22566 raft_consensus.cc:348] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } }
I20260923 22:11:47.471279 22566 raft_consensus.cc:374] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:47.471304 22566 raft_consensus.cc:729] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a052243e97bf4ecab149ff9c00d270e7, State: Initialized, Role: FOLLOWER
I20260923 22:11:47.471392 22566 consensus_queue.cc:260] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } }
I20260923 22:11:47.471436 22566 raft_consensus.cc:388] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:47.471457 22566 raft_consensus.cc:482] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:47.471479 22566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:47.471858 22566 raft_consensus.cc:504] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } }
I20260923 22:11:47.471953 22566 leader_election.cc:302] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a052243e97bf4ecab149ff9c00d270e7; no voters:
I20260923 22:11:47.472085 22566 leader_election.cc:288] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:47.472132 22569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:47.472242 22569 raft_consensus.cc:686] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [term 1 LEADER]: Becoming Leader. State: Replica: a052243e97bf4ecab149ff9c00d270e7, State: Running, Role: LEADER
I20260923 22:11:47.472322 22566 sys_catalog.cc:564] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:47.472378 22569 consensus_queue.cc:237] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } }
I20260923 22:11:47.472937 22570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a052243e97bf4ecab149ff9c00d270e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } } }
I20260923 22:11:47.473021 22570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:47.473251 22578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:47.473332 22571 sys_catalog.cc:455] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a052243e97bf4ecab149ff9c00d270e7. Latest consensus state: current_term: 1 leader_uuid: "a052243e97bf4ecab149ff9c00d270e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a052243e97bf4ecab149ff9c00d270e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44517 } } }
I20260923 22:11:47.473416 22571 sys_catalog.cc:458] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:47.473694 22578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:47.474992 22578 catalog_manager.cc:1347] Generated new cluster ID: 7113a776f8f14acfaef45349541a7bdf
I20260923 22:11:47.475039 22578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:47.475581 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22498
I20260923 22:11:47.475688 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:47.475857 1670 external_mini_cluster.cc:1428] Setting key 196317f3825c7fd897d9f8f0dfad92b5
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:47.487805 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:47.479285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:43907 (local address 127.0.0.1:44517)
0923 22:11:47.479439 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:47.479448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:47.479645 (+ 197us) server_negotiation.cc:407] Connection header received
0923 22:11:47.479788 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:47.479793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:47.479833 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:47.479983 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:47.480592 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.481618 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.482354 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.482720 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.482820 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:47.484855 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:47.484859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:47.484860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:47.484870 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:47.486405 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.486753 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.486757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.486757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.486830 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.487037 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.487038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.487038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.487286 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:47.487435 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:47.487529 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:11:47.487615 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260923 22:11:47.488504 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:47.496131 22578 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:47.496718 22578 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:47.497216 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:47.491412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:34601 (local address 127.0.0.1:44517)
0923 22:11:47.491441 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:47.491443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:47.491582 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:11:47.491642 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:47.491643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:47.491652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:47.491700 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:47.492152 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.492577 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.493221 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:47.493425 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:47.493508 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:47.494737 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:47.494738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:47.494739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:47.494756 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:47.496174 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.496507 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.496508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.496508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.496554 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:47.496765 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:47.496766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:47.496766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:47.496864 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:47.496987 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:47.497079 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 22:11:47.497131 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:11:47.500789 22578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a052243e97bf4ecab149ff9c00d270e7: Generated new TSK 0
I20260923 22:11:47.500928 22578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:47.503221 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22498
W20260923 22:11:47.506943 22483 connection.cc:535] client connection to 127.0.0.1:44517 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44517): Connection reset by peer (error 104)
2026-09-23T22:11:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22597](info): set up 2 sockets
Sep 23 22:11:47 dist-test-slave-npvd krb5kdc[22597](info): commencing operation
krb5kdc: starting...
W20260923 22:11:48.434995 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.915s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:48 dist-test-slave-npvd krb5kdc[22597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:48.538779 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:48.611241 22610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:48.611438 22610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:48.611470 22610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:48.612499 22610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:48.612558 22610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:48.612579 22610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:48.612603 22610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:48.612629 22610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:48.613838 22610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:48.614068 22610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:48.614252 22610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:48.618494 22618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:48.618521 22616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:48.618525 22615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:48.618553 22610 server_base.cc:1034] running on GCE node
I20260923 22:11:48.618942 22610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:48.619262 22610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:48.620426 22610 hybrid_clock.cc:648] HybridClock initialized: now 1790201508620406 us; error 37 us; skew 500 ppm
Sep 23 22:11:48 dist-test-slave-npvd krb5kdc[22597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:48.622689 22610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:48.623054 22610 webserver.cc:460] Webserver started at http://127.0.0.1:33847/ using document root <none> and password file <none>
I20260923 22:11:48.623298 22610 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:48.623340 22610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:48.623467 22610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:48.624330 22610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "e57dcb93d10445298a4ccdbfd21cfbd2"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-npvd"
server_key: "18d1863aa9390a0231096caecca3697e"
server_key_iv: "9e01961d777fc102aca72bb156c96de5"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.624641 22610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "e57dcb93d10445298a4ccdbfd21cfbd2"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-npvd"
server_key: "18d1863aa9390a0231096caecca3697e"
server_key_iv: "9e01961d777fc102aca72bb156c96de5"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.625773 22610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:48.626374 22625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:48.626518 22610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:48.626608 22610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "e57dcb93d10445298a4ccdbfd21cfbd2"
format_stamp: "Formatted at 2026-09-23 22:11:48 on dist-test-slave-npvd"
server_key: "18d1863aa9390a0231096caecca3697e"
server_key_iv: "9e01961d777fc102aca72bb156c96de5"
server_key_version: "encryptionkey@0"
I20260923 22:11:48.626683 22610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:48.671061 22610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:48.673170 22610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:48.673344 22610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:48.677043 22610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42783
I20260923 22:11:48.677091 22677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42783 every 8 connection(s)
I20260923 22:11:48.677431 22610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:48.678373 22678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:48.680648 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2: Bootstrap starting.
I20260923 22:11:48.681185 22678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:48.681455 22678 log.cc:824] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:48.682075 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2: No bootstrap required, opened a new log
I20260923 22:11:48.683037 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22610
I20260923 22:11:48.683224 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:48.683442 1670 external_mini_cluster.cc:1428] Setting key 32fbac10831320281b234684e6894354
I20260923 22:11:48.683835 22678 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } }
I20260923 22:11:48.684023 22678 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:48.684098 22678 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57dcb93d10445298a4ccdbfd21cfbd2, State: Initialized, Role: FOLLOWER
I20260923 22:11:48.684255 22678 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } }
I20260923 22:11:48.684334 22678 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:48.684365 22678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:48.684401 22678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:48.685389 22678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } }
I20260923 22:11:48.685483 22678 leader_election.cc:302] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57dcb93d10445298a4ccdbfd21cfbd2; no voters:
I20260923 22:11:48.685633 22678 leader_election.cc:288] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:48.685707 22683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:48.685843 22678 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:48.685933 22683 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [term 1 LEADER]: Becoming Leader. State: Replica: e57dcb93d10445298a4ccdbfd21cfbd2, State: Running, Role: LEADER
I20260923 22:11:48.686044 22683 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } }
I20260923 22:11:48.686401 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } } }
I20260923 22:11:48.686496 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:48.686631 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57dcb93d10445298a4ccdbfd21cfbd2. Latest consensus state: current_term: 1 leader_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57dcb93d10445298a4ccdbfd21cfbd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42783 } } }
I20260923 22:11:48.686691 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:48.686808 22692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:48.687454 22692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:48.689172 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:48.684001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:36727 (local address 127.0.0.1:42783)
0923 22:11:48.684181 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:48.684189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:48.684219 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:11:48.684325 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:48.684331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:48.684387 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:48.684536 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:48.684846 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.686147 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.687211 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.687759 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.687858 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:48.688521 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:48.688525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:48.688525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:48.688716 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:48.688748 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:48.688761 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:48.688838 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:48.688939 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 22:11:48.688965 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260923 22:11:48.690265 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:48.692140 22692 catalog_manager.cc:1347] Generated new cluster ID: 4be1737c721643d49e132fa275b75b26
I20260923 22:11:48.692195 22692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:48.694583 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:48.691487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:49969 (local address 127.0.0.1:42783)
0923 22:11:48.691525 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:48.691527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:48.691627 (+ 100us) server_negotiation.cc:407] Connection header received
0923 22:11:48.691700 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:48.691701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:48.691711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:48.691758 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:48.691908 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.692491 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.693380 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:48.693655 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:48.693688 (+ 33us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:48.694335 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:48.694336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:48.694337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:48.694341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:48.694353 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:48.694357 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:48.694389 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:48.694509 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 22:11:48.694520 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:11:48.709151 22692 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:48.709637 22692 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:48.713828 22692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e57dcb93d10445298a4ccdbfd21cfbd2: Generated new TSK 0
I20260923 22:11:48.713979 22692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:48.721007 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22610
W20260923 22:11:48.724818 22595 connection.cc:535] client connection to 127.0.0.1:42783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42783): Connection reset by peer (error 104)
2026-09-23T22:11:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:48 dist-test-slave-npvd krb5kdc[22709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:48 dist-test-slave-npvd krb5kdc[22709](info): set up 2 sockets
Sep 23 22:11:48 dist-test-slave-npvd krb5kdc[22709](info): commencing operation
krb5kdc: starting...
W20260923 22:11:49.650851 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:49 dist-test-slave-npvd krb5kdc[22709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:49.755964 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:49.828140 22722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:49.828305 22722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:49.828325 22722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:49.829296 22722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:49.829329 22722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:49.829344 22722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:49.829361 22722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:49.829373 22722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:49.830649 22722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:49.830798 22722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:49.830931 22722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:49.835106 22727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:49.835213 22722 server_base.cc:1034] running on GCE node
W20260923 22:11:49.835147 22728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:49.835115 22730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:49.835633 22722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:49.835858 22722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:49.836989 22722 hybrid_clock.cc:648] HybridClock initialized: now 1790201509836971 us; error 32 us; skew 500 ppm
Sep 23 22:11:49 dist-test-slave-npvd krb5kdc[22709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201509, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:49.839121 22722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:49.839661 22722 webserver.cc:460] Webserver started at http://127.0.0.1:46515/ using document root <none> and password file <none>
I20260923 22:11:49.839905 22722 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:49.839977 22722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:49.840130 22722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:49.841099 22722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "3af352e9823d437f88a29d07677df240"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-npvd"
server_key: "8886f001e128f902b78a7058099563b3"
server_key_iv: "9ecfdb28281c161a59eb629d90d49d3b"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.841442 22722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "3af352e9823d437f88a29d07677df240"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-npvd"
server_key: "8886f001e128f902b78a7058099563b3"
server_key_iv: "9ecfdb28281c161a59eb629d90d49d3b"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.842645 22722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:11:49.843291 22737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:49.843446 22722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:49.843535 22722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "3af352e9823d437f88a29d07677df240"
format_stamp: "Formatted at 2026-09-23 22:11:49 on dist-test-slave-npvd"
server_key: "8886f001e128f902b78a7058099563b3"
server_key_iv: "9ecfdb28281c161a59eb629d90d49d3b"
server_key_version: "encryptionkey@0"
I20260923 22:11:49.843616 22722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:49.846434 22722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:49.848050 22722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:49.848201 22722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:49.852077 22722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40537
I20260923 22:11:49.852102 22789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40537 every 8 connection(s)
I20260923 22:11:49.852381 22722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:49.853118 22790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:49.854789 22790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240: Bootstrap starting.
I20260923 22:11:49.855285 22790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:49.855525 22790 log.cc:824] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:49.856103 22790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240: No bootstrap required, opened a new log
I20260923 22:11:49.857434 22790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } }
I20260923 22:11:49.857542 22790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:49.857561 22790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af352e9823d437f88a29d07677df240, State: Initialized, Role: FOLLOWER
I20260923 22:11:49.857719 22790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } }
I20260923 22:11:49.857776 22790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:49.857795 22790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:49.857817 22790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:49.858242 22790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } }
I20260923 22:11:49.858310 22790 leader_election.cc:302] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3af352e9823d437f88a29d07677df240; no voters:
I20260923 22:11:49.858481 22790 leader_election.cc:288] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:49.858526 22793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:49.858682 22790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:49.858692 22793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [term 1 LEADER]: Becoming Leader. State: Replica: 3af352e9823d437f88a29d07677df240, State: Running, Role: LEADER
I20260923 22:11:49.858780 22793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } }
I20260923 22:11:49.859310 22794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3af352e9823d437f88a29d07677df240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } } }
I20260923 22:11:49.859380 22795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3af352e9823d437f88a29d07677df240. Latest consensus state: current_term: 1 leader_uuid: "3af352e9823d437f88a29d07677df240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af352e9823d437f88a29d07677df240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40537 } } }
I20260923 22:11:49.859390 22794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:49.859440 22795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:49.859484 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22722
I20260923 22:11:49.859601 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:49.859675 22802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:49.859794 1670 external_mini_cluster.cc:1428] Setting key a2acda2bcb02d3289da05a7223bf4999
I20260923 22:11:49.860209 22802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:49.861657 22802 catalog_manager.cc:1347] Generated new cluster ID: b1027488717247d2be62deae1527ad0f
I20260923 22:11:49.861713 22802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:49.864583 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:49.860401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:53795 (local address 127.0.0.1:40537)
0923 22:11:49.860950 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:49.860957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:49.863367 (+ 2410us) server_negotiation.cc:407] Connection header received
0923 22:11:49.863465 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:49.863488 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:49.863527 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:49.863670 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:49.864243 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:49.864246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:49.864247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:49.864250 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:49.864278 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:49.864290 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:49.864364 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:49.864398 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 22:11:49.864435 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":78,"threads_started":1}
I20260923 22:11:49.865309 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:49.867540 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:49.866690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:57703 (local address 127.0.0.1:40537)
0923 22:11:49.866715 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:49.866716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:49.866746 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:11:49.866820 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:49.866821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:49.866824 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:49.866872 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:49.867361 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:49.867362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:49.867362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:49.867365 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:49.867377 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:49.867381 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:49.867427 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:49.867482 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:11:49.867492 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260923 22:11:49.871556 22802 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:49.871673 22802 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:49.874459 22802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3af352e9823d437f88a29d07677df240: Generated new TSK 0
I20260923 22:11:49.874586 22802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:49.879073 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22722
2026-09-23T22:11:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1159 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:49 dist-test-slave-npvd krb5kdc[22821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:49 dist-test-slave-npvd krb5kdc[22821](info): set up 2 sockets
Sep 23 22:11:49 dist-test-slave-npvd krb5kdc[22821](info): commencing operation
krb5kdc: starting...
W20260923 22:11:50.815442 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:50 dist-test-slave-npvd krb5kdc[22821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:50.915071 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:50.985850 22834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:50.986032 22834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:50.986063 22834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:50.987022 22834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:50.987080 22834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:50.987102 22834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:50.987118 22834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:50.987143 22834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:50.988422 22834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:50.988643 22834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:50.988829 22834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:11:50.993242 22834 server_base.cc:1034] running on GCE node
W20260923 22:11:50.993207 22840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:50.993206 22842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:50.993306 22839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:50.993607 22834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:50.993849 22834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:50.994987 22834 hybrid_clock.cc:648] HybridClock initialized: now 1790201510994970 us; error 33 us; skew 500 ppm
Sep 23 22:11:50 dist-test-slave-npvd krb5kdc[22821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:50.997169 22834 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:50.997545 22834 webserver.cc:460] Webserver started at http://127.0.0.1:46487/ using document root <none> and password file <none>
I20260923 22:11:50.997772 22834 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:50.997843 22834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:50.997975 22834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:50.998895 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "1884ca04f4a146c1a288ac72dc489dff"
format_stamp: "Formatted at 2026-09-23 22:11:50 on dist-test-slave-npvd"
server_key: "422f70b235deae29fb1ac4ef1483775e"
server_key_iv: "c6c53c7ea5cd612df2c548e52c6e8d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:50.999315 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "1884ca04f4a146c1a288ac72dc489dff"
format_stamp: "Formatted at 2026-09-23 22:11:50 on dist-test-slave-npvd"
server_key: "422f70b235deae29fb1ac4ef1483775e"
server_key_iv: "c6c53c7ea5cd612df2c548e52c6e8d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:51.000593 22834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 22:11:51.001250 22849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:51.001405 22834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:51.001498 22834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "1884ca04f4a146c1a288ac72dc489dff"
format_stamp: "Formatted at 2026-09-23 22:11:50 on dist-test-slave-npvd"
server_key: "422f70b235deae29fb1ac4ef1483775e"
server_key_iv: "c6c53c7ea5cd612df2c548e52c6e8d86"
server_key_version: "encryptionkey@0"
I20260923 22:11:51.001575 22834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:51.017781 22834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:51.019362 22834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:51.019508 22834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:51.022686 22834 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44333
I20260923 22:11:51.022737 22901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44333 every 8 connection(s)
I20260923 22:11:51.023012 22834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:51.023916 22902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:51.025641 22902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff: Bootstrap starting.
I20260923 22:11:51.026110 22902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:51.026346 22902 log.cc:824] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:51.026907 22902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff: No bootstrap required, opened a new log
I20260923 22:11:51.028240 22902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } }
I20260923 22:11:51.028348 22902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:51.028366 22902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1884ca04f4a146c1a288ac72dc489dff, State: Initialized, Role: FOLLOWER
I20260923 22:11:51.028549 22902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } }
I20260923 22:11:51.028604 22902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:51.028604 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22834
I20260923 22:11:51.028647 22902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:51.028690 22902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:51.028703 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:51.028861 1670 external_mini_cluster.cc:1428] Setting key 68055a981ff48403d130eec53ea95d74
I20260923 22:11:51.029299 22902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } }
I20260923 22:11:51.029407 22902 leader_election.cc:302] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1884ca04f4a146c1a288ac72dc489dff; no voters:
I20260923 22:11:51.029560 22902 leader_election.cc:288] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:51.029594 22907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:51.029692 22907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [term 1 LEADER]: Becoming Leader. State: Replica: 1884ca04f4a146c1a288ac72dc489dff, State: Running, Role: LEADER
I20260923 22:11:51.029798 22902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:51.029824 22907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } }
I20260923 22:11:51.030313 22908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1884ca04f4a146c1a288ac72dc489dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } } }
I20260923 22:11:51.030437 22908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:51.030727 22909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1884ca04f4a146c1a288ac72dc489dff. Latest consensus state: current_term: 1 leader_uuid: "1884ca04f4a146c1a288ac72dc489dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1884ca04f4a146c1a288ac72dc489dff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44333 } } }
I20260923 22:11:51.030859 22909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:51.031023 22916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:51.031580 22916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:51.033274 22916 catalog_manager.cc:1347] Generated new cluster ID: 638bc1d910ff4eb2a2ce9b5e4f839a44
I20260923 22:11:51.033322 22916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:51 dist-test-slave-npvd krb5kdc[22821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 22:11:51.038864 22916 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:51.039543 22916 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:51.040410 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:51.029315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:38005 (local address 127.0.0.1:44333)
0923 22:11:51.029476 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:51.029486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:51.029521 (+ 35us) server_negotiation.cc:407] Connection header received
0923 22:11:51.029649 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:51.029655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:51.029715 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:51.030663 (+ 948us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:51.030785 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.032244 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.033229 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.033749 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.033837 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:51.036364 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:51.036368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:51.036368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:51.036378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:51.038621 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.039050 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.039053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.039053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.039156 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.039438 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.039439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.039439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.039684 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:51.039835 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:51.039974 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 22:11:51.040157 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260923 22:11:51.041076 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:51.046303 22916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1884ca04f4a146c1a288ac72dc489dff: Generated new TSK 0
I20260923 22:11:51.046455 22916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:51.048305 22906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:51.042364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:60021 (local address 127.0.0.1:44333)
0923 22:11:51.042387 (+ 23us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:51.042388 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:51.042543 (+ 155us) server_negotiation.cc:407] Connection header received
0923 22:11:51.042612 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:51.042612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:51.042625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:51.042646 (+ 21us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:51.043201 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.043819 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.044651 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:51.044917 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:51.044951 (+ 34us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:51.046067 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:51.046068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:51.046068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:51.046074 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:51.047327 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.047637 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.047638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.047638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.047684 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:51.047888 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:51.047889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:51.047890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:51.047981 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:51.048071 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:51.048178 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 22:11:51.048231 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10}
I20260923 22:11:51.049914 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22834
W20260923 22:11:51.053669 22819 connection.cc:535] client connection to 127.0.0.1:44333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44333): Connection reset by peer (error 104)
2026-09-23T22:11:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:51 dist-test-slave-npvd krb5kdc[22934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:51 dist-test-slave-npvd krb5kdc[22934](info): set up 2 sockets
Sep 23 22:11:51 dist-test-slave-npvd krb5kdc[22934](info): commencing operation
krb5kdc: starting...
W20260923 22:11:51.978658 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.912s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:52 dist-test-slave-npvd krb5kdc[22934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:52.078779 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:52.147360 22947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:52.147603 22947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:52.147631 22947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:52.148581 22947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:52.148641 22947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:52.148662 22947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:52.148686 22947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:52.148713 22947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:52.150075 22947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:52.150329 22947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:52.150542 22947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:52.154481 22955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:52.154485 22952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:52.154511 22953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:52.154513 22947 server_base.cc:1034] running on GCE node
I20260923 22:11:52.154843 22947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:52.155045 22947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:52.156239 22947 hybrid_clock.cc:648] HybridClock initialized: now 1790201512156220 us; error 29 us; skew 500 ppm
Sep 23 22:11:52 dist-test-slave-npvd krb5kdc[22934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:52.158392 22947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:52.158711 22947 webserver.cc:460] Webserver started at http://127.0.0.1:38729/ using document root <none> and password file <none>
I20260923 22:11:52.158883 22947 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:52.158922 22947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:52.159016 22947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:52.160003 22947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "bb1cf780af514021a5bad0b7b114c228"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-npvd"
server_key: "f2013db186c9daf2666348b1018d1139"
server_key_iv: "d773ca7164a3b96419eda302ed3c931d"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.160297 22947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "bb1cf780af514021a5bad0b7b114c228"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-npvd"
server_key: "f2013db186c9daf2666348b1018d1139"
server_key_iv: "d773ca7164a3b96419eda302ed3c931d"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.161433 22947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:52.161988 22962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:52.162159 22947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:52.162230 22947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "bb1cf780af514021a5bad0b7b114c228"
format_stamp: "Formatted at 2026-09-23 22:11:52 on dist-test-slave-npvd"
server_key: "f2013db186c9daf2666348b1018d1139"
server_key_iv: "d773ca7164a3b96419eda302ed3c931d"
server_key_version: "encryptionkey@0"
I20260923 22:11:52.162273 22947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:52.180608 22947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:52.182446 22947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:52.182538 22947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:52.185667 22947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33551
I20260923 22:11:52.185721 23014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33551 every 8 connection(s)
I20260923 22:11:52.185974 22947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:52.186774 23015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:52.188753 23015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228: Bootstrap starting.
I20260923 22:11:52.189285 23015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:52.189561 23015 log.cc:824] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:52.190215 23015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228: No bootstrap required, opened a new log
I20260923 22:11:52.191635 23015 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260923 22:11:52.191774 23015 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:52.191854 23015 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1cf780af514021a5bad0b7b114c228, State: Initialized, Role: FOLLOWER
I20260923 22:11:52.192006 23015 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260923 22:11:52.192097 23015 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:52.192135 23015 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:52.192178 23015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:52.192505 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 22947
I20260923 22:11:52.192627 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:52.192682 23015 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260923 22:11:52.192790 23015 leader_election.cc:302] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1cf780af514021a5bad0b7b114c228; no voters:
I20260923 22:11:52.192833 1670 external_mini_cluster.cc:1428] Setting key d82b179bace3f0d84c49629b2ba73b13
I20260923 22:11:52.192977 23015 leader_election.cc:288] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:52.193123 23019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:52.193194 23015 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:52.193271 23019 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [term 1 LEADER]: Becoming Leader. State: Replica: bb1cf780af514021a5bad0b7b114c228, State: Running, Role: LEADER
I20260923 22:11:52.193348 23019 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260923 22:11:52.193670 23019 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1cf780af514021a5bad0b7b114c228. Latest consensus state: current_term: 1 leader_uuid: "bb1cf780af514021a5bad0b7b114c228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } } }
I20260923 22:11:52.193739 23019 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:52.194006 23021 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1cf780af514021a5bad0b7b114c228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1cf780af514021a5bad0b7b114c228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } } }
I20260923 22:11:52.194092 23028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:52.194150 23021 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:52.194617 23028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:52.195950 23028 catalog_manager.cc:1347] Generated new cluster ID: 6842cf3116ba4a2fb27e267091113ac3
I20260923 22:11:52.195992 23028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:52.198297 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:52.193214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:36867 (local address 127.0.0.1:33551)
0923 22:11:52.193641 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:52.193650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:52.193686 (+ 36us) server_negotiation.cc:407] Connection header received
0923 22:11:52.193800 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:52.193807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:52.193863 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:52.194557 (+ 694us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:52.194664 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.195940 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.196614 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.196990 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.197117 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:52.197713 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:52.197716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:52.197717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:52.197881 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:52.197905 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:52.197914 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:52.197993 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:52.198063 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 22:11:52.198169 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":48,"threads_started":1}
I20260923 22:11:52.199019 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:52.205794 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:52.200359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:35503 (local address 127.0.0.1:33551)
0923 22:11:52.200405 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:52.200406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:52.203489 (+ 3083us) server_negotiation.cc:407] Connection header received
0923 22:11:52.203554 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:52.203555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:52.203565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:52.203608 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:52.203715 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.204123 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.204757 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:52.204965 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:52.205032 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:52.205588 (+ 556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:52.205589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:52.205589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:52.205593 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:52.205604 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:52.205607 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:52.205649 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:52.205704 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 22:11:52.205718 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 22:11:52.211813 23028 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:52.212363 23028 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:52.215237 23028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1cf780af514021a5bad0b7b114c228: Generated new TSK 0
I20260923 22:11:52.215368 23028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:52.219368 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 22947
W20260923 22:11:52.223038 22932 connection.cc:535] client connection to 127.0.0.1:33551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33551): Connection reset by peer (error 104)
2026-09-23T22:11:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:52 dist-test-slave-npvd krb5kdc[23046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:52 dist-test-slave-npvd krb5kdc[23046](info): set up 2 sockets
Sep 23 22:11:52 dist-test-slave-npvd krb5kdc[23046](info): commencing operation
krb5kdc: starting...
W20260923 22:11:53.134666 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.898s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:53 dist-test-slave-npvd krb5kdc[23046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 22:11:53.231834 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:53.301769 23059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:53.301965 23059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:53.301997 23059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:53.302958 23059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:53.303015 23059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:53.303035 23059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:53.303054 23059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:53.303079 23059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:53.304283 23059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:53.304507 23059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:53.304692 23059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:53.308837 23065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:53.308933 23059 server_base.cc:1034] running on GCE node
W20260923 22:11:53.308830 23064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:53.308830 23067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:53.309201 23059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:53.309505 23059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:53.310669 23059 hybrid_clock.cc:648] HybridClock initialized: now 1790201513310714 us; error 85 us; skew 500 ppm
Sep 23 22:11:53 dist-test-slave-npvd krb5kdc[23046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:53.316082 23059 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 22:11:53.316419 23059 webserver.cc:460] Webserver started at http://127.0.0.1:33621/ using document root <none> and password file <none>
I20260923 22:11:53.316625 23059 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:53.316687 23059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:53.316821 23059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:53.317685 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "22e9af373b5f495d8b6ede933bcc2878"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-npvd"
server_key: "f48736eccd937a57e388c3bee87044d9"
server_key_iv: "4696cc8e52f6b99f09f5cd78063c4a77"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.318022 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "22e9af373b5f495d8b6ede933bcc2878"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-npvd"
server_key: "f48736eccd937a57e388c3bee87044d9"
server_key_iv: "4696cc8e52f6b99f09f5cd78063c4a77"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.319110 23059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:53.319803 23074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:53.319963 23059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:53.320091 23059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "22e9af373b5f495d8b6ede933bcc2878"
format_stamp: "Formatted at 2026-09-23 22:11:53 on dist-test-slave-npvd"
server_key: "f48736eccd937a57e388c3bee87044d9"
server_key_iv: "4696cc8e52f6b99f09f5cd78063c4a77"
server_key_version: "encryptionkey@0"
I20260923 22:11:53.320163 23059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:53.322849 23059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:53.324757 23059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:53.324877 23059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:53.328073 23059 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36983
I20260923 22:11:53.328121 23126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36983 every 8 connection(s)
I20260923 22:11:53.328392 23059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:53.329173 23127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:53.331039 23127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: Bootstrap starting.
I20260923 22:11:53.331557 23127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:53.331794 23127 log.cc:824] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:53.332331 23127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: No bootstrap required, opened a new log
I20260923 22:11:53.333659 23127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260923 22:11:53.333775 23127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:53.333830 23127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e9af373b5f495d8b6ede933bcc2878, State: Initialized, Role: FOLLOWER
I20260923 22:11:53.333979 23127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260923 22:11:53.334036 23127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:53.334056 23127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:53.334117 23127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:53.334590 23127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260923 22:11:53.334687 23127 leader_election.cc:302] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22e9af373b5f495d8b6ede933bcc2878; no voters:
I20260923 22:11:53.334882 23127 leader_election.cc:288] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:53.334911 23130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:53.335055 23130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [term 1 LEADER]: Becoming Leader. State: Replica: 22e9af373b5f495d8b6ede933bcc2878, State: Running, Role: LEADER
I20260923 22:11:53.335111 23127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:53.335105 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23059
I20260923 22:11:53.335183 23130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260923 22:11:53.335238 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:53.335433 1670 external_mini_cluster.cc:1428] Setting key dead1cc6e7b9507dc9a2e994c25a6ef3
I20260923 22:11:53.335697 23131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22e9af373b5f495d8b6ede933bcc2878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } } }
I20260923 22:11:53.335772 23131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:53.335990 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22e9af373b5f495d8b6ede933bcc2878. Latest consensus state: current_term: 1 leader_uuid: "22e9af373b5f495d8b6ede933bcc2878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22e9af373b5f495d8b6ede933bcc2878" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } } }
I20260923 22:11:53.336069 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878 [sys.catalog]: This master's current role is: LEADER
W20260923 22:11:53.336840 23147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:11:53.336910 23147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:11:53.336966 23140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:53.337112 23140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:53.338339 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:53.335823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:37467 (local address 127.0.0.1:36983)
0923 22:11:53.336640 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:53.336649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:53.336683 (+ 34us) server_negotiation.cc:407] Connection header received
0923 22:11:53.336787 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:53.336793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:53.336821 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:53.337733 (+ 912us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:53.337960 (+ 227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:53.337962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:53.337962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:53.337964 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:53.337984 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:53.337992 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:53.338064 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:53.338137 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:11:53.338177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":66,"threads_started":1}
I20260923 22:11:53.338739 23140 catalog_manager.cc:1347] Generated new cluster ID: 95a8db2f472f40feafa22b8bf98d6d6a
I20260923 22:11:53.338785 23140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:53.339094 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:53.341380 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:53.340416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.107:54755 (local address 127.0.0.1:36983)
0923 22:11:53.340449 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:53.340451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:53.340563 (+ 112us) server_negotiation.cc:407] Connection header received
0923 22:11:53.340644 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:53.340644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:53.340649 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:53.340709 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:53.341172 (+ 463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:53.341173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:53.341173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:53.341177 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:53.341187 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:53.341191 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:53.341238 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:53.341284 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:11:53.341296 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:53.356042 23140 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:53.356217 23140 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:53.363720 23140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22e9af373b5f495d8b6ede933bcc2878: Generated new TSK 0
I20260923 22:11:53.363864 23140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:53.368045 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23059
2026-09-23T22:11:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1148 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:53 dist-test-slave-npvd krb5kdc[23158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:53 dist-test-slave-npvd krb5kdc[23158](info): set up 2 sockets
Sep 23 22:11:53 dist-test-slave-npvd krb5kdc[23158](info): commencing operation
krb5kdc: starting...
W20260923 22:11:54.294857 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.910s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:54.398483 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46825
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:46825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:54.472940 23171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:54.473153 23171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:54.473191 23171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:54.474292 23171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:54.474362 23171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:54.474390 23171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:54.474418 23171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:54.474450 23171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:54.475935 23171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:46825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:46825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:54.476181 23171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:54.476402 23171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:54.481083 23179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:54.481233 23171 server_base.cc:1034] running on GCE node
W20260923 22:11:54.481083 23176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:54.481109 23177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:54.481511 23171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:54.481781 23171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:54.482988 23171 hybrid_clock.cc:648] HybridClock initialized: now 1790201514482938 us; error 39 us; skew 500 ppm
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:54.485395 23171 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:54.485795 23171 webserver.cc:460] Webserver started at http://127.1.161.190:39203/ using document root <none> and password file <none>
I20260923 22:11:54.486012 23171 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:54.486078 23171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:54.486225 23171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:54.487150 23171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "88d0bf5458d64ec193d7b238394a1e7e"
format_stamp: "Formatted at 2026-09-23 22:11:54 on dist-test-slave-npvd"
server_key: "c04fbe796c0457cc47f8ac73ae3d7ae5"
server_key_iv: "6e828432464b65516d045122651083d3"
server_key_version: "encryptionkey@0"
I20260923 22:11:54.487545 23171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "88d0bf5458d64ec193d7b238394a1e7e"
format_stamp: "Formatted at 2026-09-23 22:11:54 on dist-test-slave-npvd"
server_key: "c04fbe796c0457cc47f8ac73ae3d7ae5"
server_key_iv: "6e828432464b65516d045122651083d3"
server_key_version: "encryptionkey@0"
I20260923 22:11:54.488706 23171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:11:54.489395 23186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:54.489586 23171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:54.489684 23171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "88d0bf5458d64ec193d7b238394a1e7e"
format_stamp: "Formatted at 2026-09-23 22:11:54 on dist-test-slave-npvd"
server_key: "c04fbe796c0457cc47f8ac73ae3d7ae5"
server_key_iv: "6e828432464b65516d045122651083d3"
server_key_version: "encryptionkey@0"
I20260923 22:11:54.489769 23171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:54.505553 23171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:54.507339 23171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:54.507495 23171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:54.511080 23171 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:46825
I20260923 22:11:54.511142 23238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:46825 every 8 connection(s)
I20260923 22:11:54.511492 23171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:54.512197 23239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:54.512463 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23171
I20260923 22:11:54.512578 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:54.512784 1670 external_mini_cluster.cc:1428] Setting key ea659453462e7de66dd28659841750cf
I20260923 22:11:54.514989 23239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: Bootstrap starting.
I20260923 22:11:54.515599 23239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:54.515920 23239 log.cc:824] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:54.516602 23239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: No bootstrap required, opened a new log
I20260923 22:11:54.519009 23239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } }
I20260923 22:11:54.519129 23239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:54.519148 23239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d0bf5458d64ec193d7b238394a1e7e, State: Initialized, Role: FOLLOWER
I20260923 22:11:54.519349 23239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } }
I20260923 22:11:54.519424 23239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:54.519446 23239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:54.519511 23239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:54.520175 23239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } }
I20260923 22:11:54.520293 23239 leader_election.cc:302] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d0bf5458d64ec193d7b238394a1e7e; no voters:
I20260923 22:11:54.520467 23239 leader_election.cc:288] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:54.520526 23244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:54.520673 23244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [term 1 LEADER]: Becoming Leader. State: Replica: 88d0bf5458d64ec193d7b238394a1e7e, State: Running, Role: LEADER
I20260923 22:11:54.520781 23239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:54.520854 23244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } }
I20260923 22:11:54.521188 23244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d0bf5458d64ec193d7b238394a1e7e. Latest consensus state: current_term: 1 leader_uuid: "88d0bf5458d64ec193d7b238394a1e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } } }
I20260923 22:11:54.521198 23245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d0bf5458d64ec193d7b238394a1e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d0bf5458d64ec193d7b238394a1e7e" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 46825 } } }
I20260923 22:11:54.521337 23244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:54.521345 23245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e [sys.catalog]: This master's current role is: LEADER
W20260923 22:11:54.522684 23259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:11:54.522843 23259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:11:54.523145 23253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:54.523772 23253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:54.525233 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:54.513257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57013 (local address 127.1.161.190:46825)
0923 22:11:54.513472 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:54.513478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:54.513500 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:11:54.513865 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:54.513871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:54.513912 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:54.514016 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:54.514780 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:54.515825 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:54.516838 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:54.517232 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:54.517312 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:54.520032 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:54.520035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:54.520035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:54.520044 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:54.522142 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:54.522780 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:54.522785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:54.522785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:54.522968 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:54.523780 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:54.523782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:54.523782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:54.524255 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:54.524839 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:54.524938 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 22:11:54.525039 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260923 22:11:54.525967 23253 catalog_manager.cc:1347] Generated new cluster ID: c7cb7968ff4043aea9ce169996ff68e7
I20260923 22:11:54.526014 23253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:54.526223 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:54.533361 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:54.527524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59727 (local address 127.1.161.190:46825)
0923 22:11:54.527559 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:54.527561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:54.527681 (+ 120us) server_negotiation.cc:407] Connection header received
0923 22:11:54.527744 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:54.527745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:54.527753 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:54.527794 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:54.528257 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:54.528930 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:54.529664 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:54.529904 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:54.529979 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:54.531043 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:54.531044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:54.531044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:54.531050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:54.532260 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:54.532606 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:54.532607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:54.532607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:54.532669 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:54.532896 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:54.532897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:54.532898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:54.532993 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:54.533087 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:54.533202 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 22:11:54.533255 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:54.534950 23253 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:54.535429 23253 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:54.537940 23253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d0bf5458d64ec193d7b238394a1e7e: Generated new TSK 0
I20260923 22:11:54.538079 23253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:54.540020 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23171
W20260923 22:11:54.544167 23156 connection.cc:535] client connection to 127.1.161.190:46825 recv error: Network error: failed to read from TLS socket (remote: 127.1.161.190:46825): Connection reset by peer (error 104)
2026-09-23T22:11:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1171 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23270](info): set up 2 sockets
Sep 23 22:11:54 dist-test-slave-npvd krb5kdc[23270](info): commencing operation
krb5kdc: starting...
W20260923 22:11:55.470849 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.914s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:55 dist-test-slave-npvd krb5kdc[23270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:55.568961 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41793
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:42125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:41793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:55.640120 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:55.640298 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:55.640321 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:55.641383 23283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:55.641418 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:55.641430 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:55.641465 23283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:55.641489 23283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:55.642788 23283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:42125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:41793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:41793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:55.643008 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:55.643240 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:55.647496 23289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:55.647554 23283 server_base.cc:1034] running on GCE node
W20260923 22:11:55.647505 23288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:55.647505 23291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:55.647871 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:55.648151 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:55.649276 23283 hybrid_clock.cc:648] HybridClock initialized: now 1790201515649268 us; error 23 us; skew 500 ppm
Sep 23 22:11:55 dist-test-slave-npvd krb5kdc[23270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:55.651587 23283 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:55.651984 23283 webserver.cc:460] Webserver started at http://127.1.161.190:45017/ using document root <none> and password file <none>
I20260923 22:11:55.652187 23283 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:55.652225 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:55.652347 23283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:55.653219 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "e1fde04373fb4720babd4ff19a26103a"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-npvd"
server_key: "44c86e8dba2d0fd9fbc43d883050f384"
server_key_iv: "ce57d37b4c6d1cca2064b2aa3c1685e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.653535 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "e1fde04373fb4720babd4ff19a26103a"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-npvd"
server_key: "44c86e8dba2d0fd9fbc43d883050f384"
server_key_iv: "ce57d37b4c6d1cca2064b2aa3c1685e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.654668 23283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:11:55.655287 23298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:55.655437 23283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:55.655498 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "e1fde04373fb4720babd4ff19a26103a"
format_stamp: "Formatted at 2026-09-23 22:11:55 on dist-test-slave-npvd"
server_key: "44c86e8dba2d0fd9fbc43d883050f384"
server_key_iv: "ce57d37b4c6d1cca2064b2aa3c1685e8"
server_key_version: "encryptionkey@0"
I20260923 22:11:55.655577 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:55.667325 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:55.668735 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:55.668870 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:55.672590 23283 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:41793
I20260923 22:11:55.672650 23350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:41793 every 8 connection(s)
I20260923 22:11:55.672894 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:55.673616 23351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:55.675439 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a: Bootstrap starting.
I20260923 22:11:55.675894 23351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:55.676096 23351 log.cc:824] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:55.676615 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a: No bootstrap required, opened a new log
I20260923 22:11:55.678038 23351 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } }
I20260923 22:11:55.678160 23351 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:55.678182 23351 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fde04373fb4720babd4ff19a26103a, State: Initialized, Role: FOLLOWER
I20260923 22:11:55.678274 23351 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } }
I20260923 22:11:55.678350 23351 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:55.678406 23351 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:55.678459 23351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:55.679270 23351 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } }
I20260923 22:11:55.679383 23351 leader_election.cc:302] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1fde04373fb4720babd4ff19a26103a; no voters:
I20260923 22:11:55.679536 23351 leader_election.cc:288] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:55.679582 23354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:55.679766 23354 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [term 1 LEADER]: Becoming Leader. State: Replica: e1fde04373fb4720babd4ff19a26103a, State: Running, Role: LEADER
I20260923 22:11:55.679811 23351 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:55.679876 23354 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } }
I20260923 22:11:55.680230 23355 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1fde04373fb4720babd4ff19a26103a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } } }
I20260923 22:11:55.680269 23356 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1fde04373fb4720babd4ff19a26103a. Latest consensus state: current_term: 1 leader_uuid: "e1fde04373fb4720babd4ff19a26103a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fde04373fb4720babd4ff19a26103a" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 41793 } } }
I20260923 22:11:55.680397 23356 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:55.680593 23355 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:55.680769 23363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:55.681252 23363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:55.682662 23363 catalog_manager.cc:1347] Generated new cluster ID: 99bdba3dc425461093892ce174ff3440
I20260923 22:11:55.682722 23363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:55.682778 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23283
I20260923 22:11:55.682888 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:55.683058 1670 external_mini_cluster.cc:1428] Setting key 6ee244a7900725f3d1ee17a21a7ad9ae
I20260923 22:11:55.687296 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:55.683594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44647 (local address 127.1.161.190:41793)
0923 22:11:55.683763 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:55.683774 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:55.683795 (+ 21us) server_negotiation.cc:407] Connection header received
0923 22:11:55.683871 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:55.683877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:55.683911 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:55.684031 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:55.684216 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.685006 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.685658 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.686040 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.686143 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:55.686732 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:55.686735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:55.686736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:55.686887 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:55.686913 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:55.686922 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:55.687005 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:55.687073 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 22:11:55.687096 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260923 22:11:55.688031 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:55.691799 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:55.689195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51595 (local address 127.1.161.190:41793)
0923 22:11:55.689229 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:55.689231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:55.689334 (+ 103us) server_negotiation.cc:407] Connection header received
0923 22:11:55.689402 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:55.689402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:55.689411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:55.689457 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:55.689582 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.690057 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.690739 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:55.690978 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:55.691055 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:55.691615 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:55.691616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:55.691617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:55.691620 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:55.691631 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:55.691635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:55.691680 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:55.691730 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:11:55.691739 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:11:55.692909 23363 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:55.693547 23363 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:55.698251 23363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1fde04373fb4720babd4ff19a26103a: Generated new TSK 0
I20260923 22:11:55.698378 23363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:55.704386 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23283
W20260923 22:11:55.707798 23268 connection.cc:535] client connection to 127.1.161.190:41793 recv error: Network error: failed to read from TLS socket (remote: 127.1.161.190:41793): Connection reset by peer (error 104)
2026-09-23T22:11:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:55 dist-test-slave-npvd krb5kdc[23382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:55 dist-test-slave-npvd krb5kdc[23382](info): set up 2 sockets
Sep 23 22:11:55 dist-test-slave-npvd krb5kdc[23382](info): commencing operation
krb5kdc: starting...
W20260923 22:11:56.626832 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.906s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:56 dist-test-slave-npvd krb5kdc[23382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:56.727945 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38627
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:56.801525 23395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:56.801679 23395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:56.801697 23395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:56.802591 23395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:56.802623 23395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:56.802635 23395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:56.802645 23395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:56.802655 23395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:56.803805 23395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:38627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:38627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:56.803959 23395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:56.804087 23395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:56.808318 23401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:56.808341 23400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:56.808434 23395 server_base.cc:1034] running on GCE node
W20260923 22:11:56.808339 23403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:56.808717 23395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:56.808993 23395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:56.810135 23395 hybrid_clock.cc:648] HybridClock initialized: now 1790201516810121 us; error 27 us; skew 500 ppm
Sep 23 22:11:56 dist-test-slave-npvd krb5kdc[23382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:56.812458 23395 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:56.812767 23395 webserver.cc:460] Webserver started at http://127.1.161.190:43547/ using document root <none> and password file <none>
I20260923 22:11:56.812947 23395 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:56.812984 23395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:56.813107 23395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:56.813921 23395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "e49251c8b2d346a6b6c4c6a362dd91a6"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-npvd"
server_key: "211408ca5cc6b2988a1d556caab84319"
server_key_iv: "2f06cfa0674f0453d893b1873382df05"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.814213 23395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "e49251c8b2d346a6b6c4c6a362dd91a6"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-npvd"
server_key: "211408ca5cc6b2988a1d556caab84319"
server_key_iv: "2f06cfa0674f0453d893b1873382df05"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.815449 23395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:11:56.816061 23410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:56.816234 23395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:56.816295 23395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "e49251c8b2d346a6b6c4c6a362dd91a6"
format_stamp: "Formatted at 2026-09-23 22:11:56 on dist-test-slave-npvd"
server_key: "211408ca5cc6b2988a1d556caab84319"
server_key_iv: "2f06cfa0674f0453d893b1873382df05"
server_key_version: "encryptionkey@0"
I20260923 22:11:56.816372 23395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:56.819097 23395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:56.820749 23395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:56.820907 23395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:56.824208 23395 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:38627
I20260923 22:11:56.824237 23462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:38627 every 8 connection(s)
I20260923 22:11:56.824527 23395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:56.825358 23463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:56.827139 23463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6: Bootstrap starting.
I20260923 22:11:56.827704 23463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:56.827955 23463 log.cc:824] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:56.828538 23463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6: No bootstrap required, opened a new log
I20260923 22:11:56.829900 23463 raft_consensus.cc:348] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } }
I20260923 22:11:56.830054 23463 raft_consensus.cc:374] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:56.830101 23463 raft_consensus.cc:729] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49251c8b2d346a6b6c4c6a362dd91a6, State: Initialized, Role: FOLLOWER
I20260923 22:11:56.830235 23463 consensus_queue.cc:260] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } }
I20260923 22:11:56.830313 23463 raft_consensus.cc:388] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:56.830354 23463 raft_consensus.cc:482] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:56.830405 23463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:56.831219 23463 raft_consensus.cc:504] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } }
I20260923 22:11:56.831321 23463 leader_election.cc:302] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e49251c8b2d346a6b6c4c6a362dd91a6; no voters:
I20260923 22:11:56.831358 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23395
I20260923 22:11:56.831467 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:56.831473 23463 leader_election.cc:288] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:56.831516 23466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:56.831651 1670 external_mini_cluster.cc:1428] Setting key 0b3e22e076ec98b2a0377f4680926933
I20260923 22:11:56.831671 23466 raft_consensus.cc:686] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [term 1 LEADER]: Becoming Leader. State: Replica: e49251c8b2d346a6b6c4c6a362dd91a6, State: Running, Role: LEADER
I20260923 22:11:56.831674 23463 sys_catalog.cc:564] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:56.831756 23466 consensus_queue.cc:237] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } }
I20260923 22:11:56.832075 23468 sys_catalog.cc:455] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e49251c8b2d346a6b6c4c6a362dd91a6. Latest consensus state: current_term: 1 leader_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } } }
I20260923 22:11:56.832094 23467 sys_catalog.cc:455] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e49251c8b2d346a6b6c4c6a362dd91a6" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 38627 } } }
I20260923 22:11:56.832140 23468 sys_catalog.cc:458] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:56.832159 23467 sys_catalog.cc:458] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:56.832554 23472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:56.833066 23472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:56.833916 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:56.832035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32993 (local address 127.1.161.190:38627)
0923 22:11:56.832267 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:56.832277 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:56.832584 (+ 307us) server_negotiation.cc:407] Connection header received
0923 22:11:56.832705 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:56.832712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:56.832741 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:56.832875 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:56.833451 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:56.833454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:56.833455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:56.833458 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:56.833491 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:56.833503 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:56.833570 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:56.833717 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 22:11:56.833750 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":59,"threads_started":1}
I20260923 22:11:56.834718 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:56.835493 23472 catalog_manager.cc:1347] Generated new cluster ID: 7eacba564d124c09b11e8841bf63aa37
I20260923 22:11:56.835543 23472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:56.842418 23472 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:56.842612 23472 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:56.843495 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:56.842518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50769 (local address 127.1.161.190:38627)
0923 22:11:56.842552 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:56.842554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:56.842677 (+ 123us) server_negotiation.cc:407] Connection header received
0923 22:11:56.842758 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:56.842759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:56.842764 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:56.842808 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:56.843305 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:56.843306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:56.843306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:56.843310 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:56.843321 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:56.843324 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:56.843373 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:56.843420 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 22:11:56.843432 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:11:56.849602 23472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e49251c8b2d346a6b6c4c6a362dd91a6: Generated new TSK 0
I20260923 22:11:56.849795 23472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:56.851562 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23395
2026-09-23T22:11:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:56 dist-test-slave-npvd krb5kdc[23494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:56 dist-test-slave-npvd krb5kdc[23494](info): set up 2 sockets
Sep 23 22:11:56 dist-test-slave-npvd krb5kdc[23494](info): commencing operation
krb5kdc: starting...
W20260923 22:11:57.782979 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.915s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:57 dist-test-slave-npvd krb5kdc[23494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:57.890106 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33333
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:57.963869 23507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:57.964053 23507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:57.964087 23507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:57.965039 23507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:57.965097 23507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:57.965119 23507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:57.965142 23507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:57.965169 23507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:57.966563 23507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:57.966817 23507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:57.967021 23507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:57.971315 23513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:57.971315 23512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:57.971443 23507 server_base.cc:1034] running on GCE node
W20260923 22:11:57.971349 23515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:57.971664 23507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:57.971921 23507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:57.973060 23507 hybrid_clock.cc:648] HybridClock initialized: now 1790201517973046 us; error 27 us; skew 500 ppm
Sep 23 22:11:57 dist-test-slave-npvd krb5kdc[23494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:57.975416 23507 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:57.975790 23507 webserver.cc:460] Webserver started at http://127.1.161.190:39941/ using document root <none> and password file <none>
I20260923 22:11:57.976042 23507 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:57.976109 23507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:57.976248 23507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:57.977202 23507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "dd92c1a3e2594ebbaf4051120a624461"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-npvd"
server_key: "3f84aeae87d65501bb67be3570c5e870"
server_key_iv: "529a9502d0b2430cbf2f07192ac6c629"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.977522 23507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "dd92c1a3e2594ebbaf4051120a624461"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-npvd"
server_key: "3f84aeae87d65501bb67be3570c5e870"
server_key_iv: "529a9502d0b2430cbf2f07192ac6c629"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.978689 23507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:57.979323 23522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:57.979473 23507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:57.979557 23507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "dd92c1a3e2594ebbaf4051120a624461"
format_stamp: "Formatted at 2026-09-23 22:11:57 on dist-test-slave-npvd"
server_key: "3f84aeae87d65501bb67be3570c5e870"
server_key_iv: "529a9502d0b2430cbf2f07192ac6c629"
server_key_version: "encryptionkey@0"
I20260923 22:11:57.979631 23507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:58.004694 23507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:58.006711 23507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:58.006865 23507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:58.010561 23507 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:33333
I20260923 22:11:58.010593 23574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:33333 every 8 connection(s)
I20260923 22:11:58.010900 23507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:58.011631 23575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:58.013382 23575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461: Bootstrap starting.
I20260923 22:11:58.013798 23575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:58.013993 23575 log.cc:824] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:58.014292 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23507
I20260923 22:11:58.014421 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:58.014585 1670 external_mini_cluster.cc:1428] Setting key 15ae8484adfc7f2b914d941f5aefc25a
I20260923 22:11:58.014704 23575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461: No bootstrap required, opened a new log
I20260923 22:11:58.016263 23575 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } }
I20260923 22:11:58.016409 23575 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:58.016443 23575 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd92c1a3e2594ebbaf4051120a624461, State: Initialized, Role: FOLLOWER
I20260923 22:11:58.016570 23575 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } }
I20260923 22:11:58.016669 23575 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:58.016698 23575 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:58.016744 23575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:58.017608 23575 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } }
I20260923 22:11:58.017730 23575 leader_election.cc:302] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd92c1a3e2594ebbaf4051120a624461; no voters:
I20260923 22:11:58.017870 23575 leader_election.cc:288] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:58.017889 23580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:58.018018 23580 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [term 1 LEADER]: Becoming Leader. State: Replica: dd92c1a3e2594ebbaf4051120a624461, State: Running, Role: LEADER
I20260923 22:11:58.018110 23575 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:58.018133 23580 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } }
I20260923 22:11:58.018421 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd92c1a3e2594ebbaf4051120a624461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } } }
I20260923 22:11:58.018507 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:58.018409 23580 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd92c1a3e2594ebbaf4051120a624461. Latest consensus state: current_term: 1 leader_uuid: "dd92c1a3e2594ebbaf4051120a624461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd92c1a3e2594ebbaf4051120a624461" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33333 } } }
I20260923 22:11:58.018664 23580 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461 [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:58.018893 23586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:58.019347 23586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:58.020820 23586 catalog_manager.cc:1347] Generated new cluster ID: 4d5357061d0044a0991b76157d6f5f18
I20260923 22:11:58.020874 23586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 22:11:58 dist-test-slave-npvd krb5kdc[23494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:11:58.025677 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:58.015031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55831 (local address 127.1.161.190:33333)
0923 22:11:58.015208 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:58.015215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:58.015246 (+ 31us) server_negotiation.cc:407] Connection header received
0923 22:11:58.016250 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:58.016254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:58.016291 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:58.016384 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:58.017103 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:58.018010 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:58.018934 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:58.019468 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:58.019829 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:58.022399 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:58.022403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:58.022404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:58.022414 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:58.024267 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:58.024614 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:58.024616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:58.024617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:58.024725 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:58.024904 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:58.024905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:58.024905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:58.025157 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:58.025302 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:58.025385 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:11:58.025507 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260923 22:11:58.026499 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:58.027926 23586 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:58.028324 23586 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:58.032634 23586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd92c1a3e2594ebbaf4051120a624461: Generated new TSK 0
I20260923 22:11:58.032763 23586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:58.037534 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:58.027877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37171 (local address 127.1.161.190:33333)
0923 22:11:58.027905 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:58.027907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:58.031425 (+ 3518us) server_negotiation.cc:407] Connection header received
0923 22:11:58.031512 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:58.031512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:58.031521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:58.031546 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:58.032204 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:58.032885 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:58.033873 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:58.034064 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:58.034119 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:58.035279 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:58.035280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:58.035280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:11:58.035287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:58.036529 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:58.036900 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:58.036901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:58.036901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:58.036945 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:11:58.037128 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:11:58.037129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:11:58.037129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:11:58.037210 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:58.037288 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:58.037430 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 22:11:58.037484 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:11:58.039300 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23507
2026-09-23T22:11:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:58 dist-test-slave-npvd krb5kdc[23606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:58 dist-test-slave-npvd krb5kdc[23606](info): set up 2 sockets
Sep 23 22:11:58 dist-test-slave-npvd krb5kdc[23606](info): commencing operation
krb5kdc: starting...
W20260923 22:11:58.962844 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.907s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:11:59 dist-test-slave-npvd krb5kdc[23606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:11:59.059595 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36115
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:35131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:36115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:11:59.131668 23619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:11:59.131848 23619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:11:59.131867 23619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:11:59.132889 23619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:11:59.132923 23619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:11:59.132936 23619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:11:59.132946 23619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:11:59.132987 23619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:11:59.134218 23619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:35131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:36115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:36115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:11:59.134416 23619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:11:59.134598 23619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:11:59.139190 23625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:59.139211 23619 server_base.cc:1034] running on GCE node
W20260923 22:11:59.139289 23624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:11:59.139390 23627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:11:59.139595 23619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:11:59.139874 23619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:11:59.141029 23619 hybrid_clock.cc:648] HybridClock initialized: now 1790201519141012 us; error 41 us; skew 500 ppm
Sep 23 22:11:59 dist-test-slave-npvd krb5kdc[23606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:11:59.148722 23619 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:11:59.149067 23619 webserver.cc:460] Webserver started at http://127.1.161.190:35621/ using document root <none> and password file <none>
I20260923 22:11:59.149221 23619 fs_manager.cc:359] Metadata directory not provided
I20260923 22:11:59.149258 23619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:11:59.149348 23619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:11:59.150243 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "1b796cc042194fea86f8092ce1130b9d"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-npvd"
server_key: "087aa46fe30084117361f21397e910e5"
server_key_iv: "6ce978effc74894d99c78b023a1f796b"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.150548 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "1b796cc042194fea86f8092ce1130b9d"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-npvd"
server_key: "087aa46fe30084117361f21397e910e5"
server_key_iv: "6ce978effc74894d99c78b023a1f796b"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.151840 23619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:11:59.152447 23634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:11:59.152612 23619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:11:59.152681 23619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "1b796cc042194fea86f8092ce1130b9d"
format_stamp: "Formatted at 2026-09-23 22:11:59 on dist-test-slave-npvd"
server_key: "087aa46fe30084117361f21397e910e5"
server_key_iv: "6ce978effc74894d99c78b023a1f796b"
server_key_version: "encryptionkey@0"
I20260923 22:11:59.152725 23619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:11:59.162143 23619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:11:59.163712 23619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:11:59.163796 23619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:11:59.166791 23619 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:36115
I20260923 22:11:59.166860 23686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:36115 every 8 connection(s)
I20260923 22:11:59.167150 23619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:11:59.168032 23687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:11:59.169919 23687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d: Bootstrap starting.
I20260923 22:11:59.170492 23687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d: Neither blocks nor log segments found. Creating new log.
I20260923 22:11:59.170781 23687 log.cc:824] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d: Log is configured to *not* fsync() on all Append() calls
I20260923 22:11:59.171506 23687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d: No bootstrap required, opened a new log
I20260923 22:11:59.172906 23687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } }
I20260923 22:11:59.173050 23687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:11:59.173098 23687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b796cc042194fea86f8092ce1130b9d, State: Initialized, Role: FOLLOWER
I20260923 22:11:59.173156 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23619
I20260923 22:11:59.173276 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:11:59.173270 23687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } }
I20260923 22:11:59.173372 23687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:11:59.173411 23687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:11:59.173465 1670 external_mini_cluster.cc:1428] Setting key 22508e45c92aae3b594bd839bdc33acf
I20260923 22:11:59.173460 23687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:11:59.174520 23687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } }
I20260923 22:11:59.174643 23687 leader_election.cc:302] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b796cc042194fea86f8092ce1130b9d; no voters:
I20260923 22:11:59.174811 23687 leader_election.cc:288] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:11:59.174863 23692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:11:59.175050 23692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [term 1 LEADER]: Becoming Leader. State: Replica: 1b796cc042194fea86f8092ce1130b9d, State: Running, Role: LEADER
I20260923 22:11:59.175064 23687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:11:59.175201 23692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } }
I20260923 22:11:59.175778 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b796cc042194fea86f8092ce1130b9d. Latest consensus state: current_term: 1 leader_uuid: "1b796cc042194fea86f8092ce1130b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } } }
I20260923 22:11:59.175904 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:59.175850 23693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b796cc042194fea86f8092ce1130b9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b796cc042194fea86f8092ce1130b9d" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 36115 } } }
I20260923 22:11:59.176106 23693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d [sys.catalog]: This master's current role is: LEADER
I20260923 22:11:59.176465 23699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:11:59.176937 23699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:11:59.178716 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:59.173949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41099 (local address 127.1.161.190:36115)
0923 22:11:59.174107 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:59.174113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:59.174135 (+ 22us) server_negotiation.cc:407] Connection header received
0923 22:11:59.174214 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:59.174219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:59.174255 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:59.174354 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:59.174624 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.175888 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.176900 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.177400 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.177469 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:59.178172 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:59.178175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:59.178175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:59.178301 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:59.178324 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:59.178333 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:59.178384 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:59.178468 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 22:11:59.178507 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260923 22:11:59.179473 23699 catalog_manager.cc:1347] Generated new cluster ID: 0f36476f65f84672a3b7ea0452f80106
I20260923 22:11:59.179515 23699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:11:59.179556 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:11:59.186383 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:11:59.183467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43725 (local address 127.1.161.190:36115)
0923 22:11:59.183502 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:11:59.183504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:11:59.183566 (+ 62us) server_negotiation.cc:407] Connection header received
0923 22:11:59.183646 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:11:59.183646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:11:59.183654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:11:59.183691 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:11:59.183829 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.184397 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.185243 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:11:59.185485 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:11:59.185551 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:11:59.186161 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:11:59.186162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:11:59.186162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:11:59.186179 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:11:59.186188 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:11:59.186191 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:11:59.186229 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 22:11:59.186295 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:11:59.186312 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 22:11:59.192358 23699 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:11:59.192759 23699 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:11:59.199208 23699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b796cc042194fea86f8092ce1130b9d: Generated new TSK 0
I20260923 22:11:59.199352 23699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:11:59.204725 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23619
2026-09-23T22:11:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1166 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:11:59 dist-test-slave-npvd krb5kdc[23718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:11:59 dist-test-slave-npvd krb5kdc[23718](info): set up 2 sockets
Sep 23 22:11:59 dist-test-slave-npvd krb5kdc[23718](info): commencing operation
krb5kdc: starting...
W20260923 22:12:00.126825 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.905s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:12:00 dist-test-slave-npvd krb5kdc[23718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:12:00.230130 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33965
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:00.299563 23731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:00.299748 23731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:00.299784 23731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:00.300900 23731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:00.300964 23731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:00.300988 23731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:00.301015 23731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:00.301046 23731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:00.302327 23731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:00.302551 23731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:00.302749 23731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:00.306962 23731 server_base.cc:1034] running on GCE node
W20260923 22:12:00.306934 23737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:00.306942 23736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:00.306934 23739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:00.307389 23731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:00.307642 23731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:00.308823 23731 hybrid_clock.cc:648] HybridClock initialized: now 1790201520308801 us; error 31 us; skew 500 ppm
Sep 23 22:12:00 dist-test-slave-npvd krb5kdc[23718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:00.311062 23731 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:12:00.311494 23731 webserver.cc:460] Webserver started at http://127.1.161.190:35719/ using document root <none> and password file <none>
I20260923 22:12:00.311653 23731 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:00.311690 23731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:00.311764 23731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:00.312561 23731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-npvd"
server_key: "9215fa9971e437a0707635b34290564a"
server_key_iv: "2c97336d73c3bc667a54013aa9c97f56"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.312832 23731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-npvd"
server_key: "9215fa9971e437a0707635b34290564a"
server_key_iv: "2c97336d73c3bc667a54013aa9c97f56"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.313902 23731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:00.314477 23746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:00.314607 23731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:00.314677 23731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4"
format_stamp: "Formatted at 2026-09-23 22:12:00 on dist-test-slave-npvd"
server_key: "9215fa9971e437a0707635b34290564a"
server_key_iv: "2c97336d73c3bc667a54013aa9c97f56"
server_key_version: "encryptionkey@0"
I20260923 22:12:00.314720 23731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:00.317420 23731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:00.319298 23731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:00.319392 23731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:00.322600 23731 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:33965
I20260923 22:12:00.322676 23798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:33965 every 8 connection(s)
I20260923 22:12:00.322963 23731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:00.323377 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23731
I20260923 22:12:00.323511 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:12:00.323722 1670 external_mini_cluster.cc:1428] Setting key b83fd0b35bce1d8a5a5c1f9968ba7c60
I20260923 22:12:00.324054 23799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:00.325981 23799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4: Bootstrap starting.
I20260923 22:12:00.326479 23799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:00.326732 23799 log.cc:824] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:00.327518 23799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4: No bootstrap required, opened a new log
I20260923 22:12:00.327862 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:00.324173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57717 (local address 127.1.161.190:33965)
0923 22:12:00.326204 (+ 2031us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:00.326210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:00.326249 (+ 39us) server_negotiation.cc:407] Connection header received
0923 22:12:00.326684 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:00.326690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:00.326711 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:00.326847 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:00.327478 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:00.327481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:00.327482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:00.327486 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:00.327522 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:00.327536 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:00.327624 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:00.327670 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:12:00.327700 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1992,"thread_start_us":68,"threads_started":1}
I20260923 22:12:00.328675 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:00.329521 23799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } }
I20260923 22:12:00.329674 23799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 22:12:00.329715 23799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4e6c570b9c4bf29cdf654c6f99c4b4, State: Initialized, Role: FOLLOWER
I20260923 22:12:00.329834 23799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } }
I20260923 22:12:00.329931 23799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 22:12:00.329970 23799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 22:12:00.330021 23799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:00.330816 23799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } }
I20260923 22:12:00.330924 23799 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c4e6c570b9c4bf29cdf654c6f99c4b4; no voters:
I20260923 22:12:00.331084 23799 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 22:12:00.331207 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:00.330217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40033 (local address 127.1.161.190:33965)
0923 22:12:00.330262 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:00.330264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:00.330380 (+ 116us) server_negotiation.cc:407] Connection header received
0923 22:12:00.330456 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:00.330457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:00.330460 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:00.330502 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:00.330976 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:00.330977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:00.330977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 22:12:00.330982 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:00.330996 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 22:12:00.331000 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:00.331051 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:00.331110 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 22:12:00.331122 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 22:12:00.331101 23809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:00.331405 23799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:00.331429 23809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [term 1 LEADER]: Becoming Leader. State: Replica: 2c4e6c570b9c4bf29cdf654c6f99c4b4, State: Running, Role: LEADER
I20260923 22:12:00.331509 23809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } }
I20260923 22:12:00.331993 23811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4e6c570b9c4bf29cdf654c6f99c4b4. Latest consensus state: current_term: 1 leader_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } } }
I20260923 22:12:00.332020 23810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c4e6c570b9c4bf29cdf654c6f99c4b4" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33965 } } }
I20260923 22:12:00.332171 23810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:00.332356 23811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:00.332399 23818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:00.332854 23818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:00.334372 23818 catalog_manager.cc:1347] Generated new cluster ID: 8bfa95e0709243a2864091094aa386cd
I20260923 22:12:00.334426 23818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:00.342293 23818 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:00.342456 23818 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:00.346370 23818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c4e6c570b9c4bf29cdf654c6f99c4b4: Generated new TSK 0
I20260923 22:12:00.346529 23818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:00.348536 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23731
2026-09-23T22:12:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1144 ms)
[----------] 18 tests from AuthTokenIssuingTest (21091 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:00 dist-test-slave-npvd krb5kdc[23830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:00 dist-test-slave-npvd krb5kdc[23830](info): set up 2 sockets
Sep 23 22:12:00 dist-test-slave-npvd krb5kdc[23830](info): commencing operation
krb5kdc: starting...
W20260923 22:12:01.270812 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.904s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:12:01.371510 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:39533
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:01.440992 23843 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:01.441171 23843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:01.441188 23843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:01.441207 23843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:01.442126 23843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:01.442157 23843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:01.442167 23843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:01.442178 23843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:01.442188 23843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:01.443503 23843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:01.443675 23843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:01.443817 23843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:01.447986 23851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.447986 23848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.448017 23849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:01.448372 23843 server_base.cc:1034] running on GCE node
I20260923 22:12:01.448602 23843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:01.448868 23843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:01.450017 23843 hybrid_clock.cc:648] HybridClock initialized: now 1790201521450007 us; error 25 us; skew 500 ppm
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:01.452416 23843 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:12:01.452833 23843 webserver.cc:460] Webserver started at http://127.1.161.190:34441/ using document root <none> and password file <none>
I20260923 22:12:01.453030 23843 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:01.453090 23843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:01.453223 23843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:01.454135 23843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "b7ea40ad8bad4418910a409b2f5ab941"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "161264b48ad553318f1d495f2859d177"
server_key_iv: "ee10bc05a805591e145e283a10afa3c9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.454481 23843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "b7ea40ad8bad4418910a409b2f5ab941"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "161264b48ad553318f1d495f2859d177"
server_key_iv: "ee10bc05a805591e145e283a10afa3c9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.455807 23843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 22:12:01.456447 23858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:01.456626 23843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:01.456720 23843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "b7ea40ad8bad4418910a409b2f5ab941"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "161264b48ad553318f1d495f2859d177"
server_key_iv: "ee10bc05a805591e145e283a10afa3c9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.456794 23843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:01.469672 23843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:01.471390 23843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:01.471562 23843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:01.475013 23843 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:39533
I20260923 22:12:01.475040 23910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:39533 every 8 connection(s)
I20260923 22:12:01.475345 23843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:01.476116 23911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:01.477357 23911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:12:01.485080 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23843
I20260923 22:12:01.485109 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.478010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.1.161.190:39533)
0923 22:12:01.478125 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.478129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.478161 (+ 32us) server_negotiation.cc:407] Connection header received
0923 22:12:01.478467 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.478469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.478482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.478546 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.478933 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.479819 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.480647 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.480953 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.481026 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.482695 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.482701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.482701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.482706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.484086 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.484360 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.484362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.484362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.484426 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.484574 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.484575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.484575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.484716 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.484762 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.484890 (+ 128us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.484972 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260923 22:12:01.485169 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:12:01.485273 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.477877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:39533 (local address 127.0.0.1:48134)
0923 22:12:01.477991 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.478016 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.478359 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.478552 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.478554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.478737 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.478860 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.478863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.479918 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.479919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.480571 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.480573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.481039 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.481040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.481091 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.481490 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.481497 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.482618 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.484145 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.484150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.484151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.484298 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.484477 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.484477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.484478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.484540 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.484766 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.484769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.484810 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.484878 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.485027 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":31,"thread_start_us":63,"threads_started":1}
I20260923 22:12:01.485342 1670 external_mini_cluster.cc:1428] Setting key 3c384e9ea0ff791ba53763750273fb5d
I20260923 22:12:01.485875 23911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:01.486136 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.486047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52150)
0923 22:12:01.486088 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.486120 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:01.486275 23860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
I20260923 22:12:01.486663 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.486561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52166)
0923 22:12:01.486612 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.486639 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:01.486763 23911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:45121: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.161.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
I20260923 22:12:01.505764 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:45121
--webserver_interface=127.1.161.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:01.535995 23911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } attempt: 1
I20260923 22:12:01.536397 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.536280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52176)
0923 22:12:01.536335 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.536369 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 22:12:01.536773 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.536679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52178)
0923 22:12:01.536726 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.536752 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:01.536918 23911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:45121: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
W20260923 22:12:01.576188 23918 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:01.576396 23918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:01.576437 23918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:01.576476 23918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:01.577486 23918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:01.577553 23918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:01.577579 23918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:01.577608 23918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:01.577637 23918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:01.578939 23918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.161.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:01.579196 23918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:01.579382 23918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:01.583789 23924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.583796 23923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.583837 23926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:01.584097 23918 server_base.cc:1034] running on GCE node
I20260923 22:12:01.584306 23918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:01.584523 23918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:01.585686 23918 hybrid_clock.cc:648] HybridClock initialized: now 1790201521585661 us; error 23 us; skew 500 ppm
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:01.587837 23918 init.cc:377] Logged in from keytab as kudu/127.1.161.189@KRBTEST.COM (short username kudu)
I20260923 22:12:01.588287 23918 webserver.cc:460] Webserver started at http://127.1.161.189:37195/ using document root <none> and password file <none>
I20260923 22:12:01.588511 23918 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:01.588579 23918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:01.588739 23918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:01.589672 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data/instance:
uuid: "2f9bc6e29b4241659ca8412aa0cc16f0"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "258a48f7adfa48aa9518ee7593572e35"
server_key_iv: "d979bfa258f7ab6bf1288453cc9bce51"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.590004 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal/instance:
uuid: "2f9bc6e29b4241659ca8412aa0cc16f0"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "258a48f7adfa48aa9518ee7593572e35"
server_key_iv: "d979bfa258f7ab6bf1288453cc9bce51"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.591359 23918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 22:12:01.592094 23933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:01.592257 23918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:01.592351 23918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "2f9bc6e29b4241659ca8412aa0cc16f0"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "258a48f7adfa48aa9518ee7593572e35"
server_key_iv: "d979bfa258f7ab6bf1288453cc9bce51"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.592424 23918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:01.600881 23918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:01.602505 23918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:01.602658 23918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:01.605077 23911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } attempt: 2
I20260923 22:12:01.605474 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.605386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52182)
0923 22:12:01.605421 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.605452 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 22:12:01.605780 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.605692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52196)
0923 22:12:01.605736 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.605758 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:12:01.605870 23911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:45121: Network error: Client connection negotiation failed: client connection to 127.1.161.189:45121: connect: Connection refused (error 111)
I20260923 22:12:01.606946 23918 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.189:45121
I20260923 22:12:01.606973 23985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.189:45121 every 8 connection(s)
I20260923 22:12:01.607323 23918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:01.608203 23986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:01.609402 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23918
I20260923 22:12:01.609496 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-1/wal/instance
I20260923 22:12:01.609706 1670 external_mini_cluster.cc:1428] Setting key 0fa062dd87d06280bf32c45fb97d041f
I20260923 22:12:01.609695 23986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.161.188@KRBTEST.COM; defaulting to no policy
I20260923 22:12:01.618496 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.610375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48140 (local address 127.1.161.190:39533)
0923 22:12:01.610444 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.610446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.610582 (+ 136us) server_negotiation.cc:407] Connection header received
0923 22:12:01.611041 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.611043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.611060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.611139 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.611957 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.612480 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.613410 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.613737 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.613823 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.615997 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.615999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.615999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.616004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.617308 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.617775 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.617776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.617777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.617836 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.618064 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.618064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.618064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.618151 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.618203 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.618396 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.618448 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 22:12:01.618774 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.610307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:39533 (local address 127.0.0.1:48140)
0923 22:12:01.610466 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.610502 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.610932 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.611250 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.611255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.611605 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.611853 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.611858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.612579 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.612579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.613318 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.613320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.613830 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.613830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.613928 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.614360 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.614374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.615909 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.617398 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.617405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.617407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.617703 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.617903 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.617904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.617904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.618010 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.618211 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.618215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.618313 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.618380 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.618476 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260923 22:12:01.619292 23986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.188@KRBTEST.COM" created.
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:01.626389 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.619504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52212)
0923 22:12:01.619552 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.619564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.619612 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.619806 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.619807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.620015 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.620075 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.620077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.621284 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.621284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.622044 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.622046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.622462 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.622464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.622539 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.622847 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.622852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.624092 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.625514 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.625515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.625515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.625652 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.625829 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.625830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.625830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.625874 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.626096 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.626097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.626128 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.626179 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.626256 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:01.626469 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.619581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52212 (local address 127.1.161.189:45121)
0923 22:12:01.619706 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.619709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.619721 (+ 12us) server_negotiation.cc:407] Connection header received
0923 22:12:01.619737 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.619738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.619747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.619798 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.620156 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.621191 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.622136 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.622392 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.622439 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.624162 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.624169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.624169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.624175 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.625426 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.625724 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.625726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.625727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.625773 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.625924 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.625924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.625925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.626031 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.626103 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.626201 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.626314 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260923 22:12:01.627115 23986 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:01.627400 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.627314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51468)
0923 22:12:01.627347 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.627383 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260923 22:12:01.627519 23937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
I20260923 22:12:01.627987 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.627895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51476)
0923 22:12:01.627941 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.627969 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:12:01.628083 23986 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:46283: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
I20260923 22:12:01.633924 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:46283
--webserver_interface=127.1.161.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:01.677268 23986 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } attempt: 1
I20260923 22:12:01.677682 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.677564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51490)
0923 22:12:01.677615 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.677653 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:12:01.678010 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.677924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51492)
0923 22:12:01.677969 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.677991 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 22:12:01.678115 23986 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:46283: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
I20260923 22:12:01.696988 23911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } attempt: 3
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:01.703042 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.697294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.1.161.189:45121)
0923 22:12:01.697345 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.697347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.697356 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:01.697427 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.697432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.697440 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.697494 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.697791 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.698214 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.698821 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.699015 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.699082 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.700822 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.700823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.700824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.700831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.702098 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.702395 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.702396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.702396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.702443 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.702615 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.702616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.702616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.702709 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.702771 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.702907 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.702968 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:01.703045 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.697247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52228)
0923 22:12:01.697297 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.697304 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.697360 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.697500 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.697501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.697648 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.697708 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.697709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.698287 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.698288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.698748 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.698749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.699078 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.699079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.699199 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.699610 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.699615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.700736 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.702172 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.702173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.702173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.702316 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.702508 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.702509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.702509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.702555 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.702773 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.702774 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.702812 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.702884 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.702965 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:01.703456 23911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:01.703748 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.703657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51500)
0923 22:12:01.703702 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.703726 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 22:12:01.704088 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.704007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51502)
0923 22:12:01.704056 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.704075 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 22:12:01.704186 23911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:46283: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
W20260923 22:12:01.705058 23993 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:01.705217 23993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:01.705266 23993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:01.705293 23993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:01.706372 23993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:01.706410 23993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:01.706422 23993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:01.706434 23993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:01.706476 23993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:01.707866 23993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:39533,127.1.161.189:45121,127.1.161.188:46283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:46283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.161.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:01.708087 23993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:01.708298 23993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:01.712450 23998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:01.712596 23993 server_base.cc:1034] running on GCE node
W20260923 22:12:01.712558 23999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:01.712558 24001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:01.712914 23993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:01.713151 23993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:01.714308 23993 hybrid_clock.cc:648] HybridClock initialized: now 1790201521714303 us; error 28 us; skew 500 ppm
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:01.716363 23993 init.cc:377] Logged in from keytab as kudu/127.1.161.188@KRBTEST.COM (short username kudu)
I20260923 22:12:01.716806 23993 webserver.cc:460] Webserver started at http://127.1.161.188:45255/ using document root <none> and password file <none>
I20260923 22:12:01.717007 23993 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:01.717068 23993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:01.717186 23993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:01.718055 23993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data/instance:
uuid: "b95360a89f19464191a3ca4eec4d2822"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "11610e880fa2fa0d926348f1885edcca"
server_key_iv: "6f0c315aa130609536655bbb3abdbbe9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.718393 23993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal/instance:
uuid: "b95360a89f19464191a3ca4eec4d2822"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "11610e880fa2fa0d926348f1885edcca"
server_key_iv: "6f0c315aa130609536655bbb3abdbbe9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.719575 23993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 22:12:01.720175 24008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:01.720364 23993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:01.720458 23993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "b95360a89f19464191a3ca4eec4d2822"
format_stamp: "Formatted at 2026-09-23 22:12:01 on dist-test-slave-npvd"
server_key: "11610e880fa2fa0d926348f1885edcca"
server_key_iv: "6f0c315aa130609536655bbb3abdbbe9"
server_key_version: "encryptionkey@0"
I20260923 22:12:01.720536 23993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:01.731729 23993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:01.733520 23993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:01.733657 23993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:01.735312 23911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } attempt: 1
I20260923 22:12:01.735643 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.735538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51510)
0923 22:12:01.735579 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.735611 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 22:12:01.735936 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.735855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51526)
0923 22:12:01.735884 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.735911 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260923 22:12:01.736034 23911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:46283: Network error: Client connection negotiation failed: client connection to 127.1.161.188:46283: connect: Connection refused (error 111)
I20260923 22:12:01.737419 23993 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.188:46283
I20260923 22:12:01.737476 24060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.188:46283 every 8 connection(s)
I20260923 22:12:01.737731 23993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:01.738492 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:01.739725 24061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:12:01.746261 23986 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } attempt: 2
I20260923 22:12:01.746884 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.740232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.1.161.190:39533)
0923 22:12:01.740344 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.740346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.740529 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:12:01.740916 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.740917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.740924 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.740975 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.741509 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.741890 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.742657 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.742820 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.742873 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.744622 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.744623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.744624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.744630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.745618 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.745965 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.745966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.745966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.746019 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.746255 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.746255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.746256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.746386 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.746438 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.746718 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.746786 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260923 22:12:01.747017 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.740198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:39533 (local address 127.0.0.1:48146)
0923 22:12:01.740410 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.740447 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.740842 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.741030 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.741033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.741237 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.741425 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.741428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.741955 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.741955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.742596 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.742598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.742891 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.742892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.742979 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.743468 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.743475 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.744540 (+ 1065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.745695 (+ 1155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.745701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.745703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.745895 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.746080 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.746080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.746081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.746179 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.746443 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.746449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.746601 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.746697 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.746824 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":141,"threads_started":1}
I20260923 22:12:01.747555 24061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:01.747576 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 23993
I20260923 22:12:01.747658 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790201274647240-1670-0/minicluster-data/master-2/wal/instance
I20260923 22:12:01.747824 1670 external_mini_cluster.cc:1428] Setting key 3b4b24a22588d027b84962dba274f6e0
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:01.754451 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.746491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51534)
0923 22:12:01.746551 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.746559 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.746596 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.746816 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.746817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.747028 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.747066 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.747068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.748178 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.748180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.748999 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.749001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.749514 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.749515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.749579 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.749918 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.749923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.751287 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.753244 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.753245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.753551 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.753774 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.753775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.753827 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.754104 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.754105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.754137 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.754257 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.754346 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 22:12:01.754561 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.747811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.1.161.189:45121)
0923 22:12:01.747843 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.747844 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.747851 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:01.747926 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.747927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.747936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.747982 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.748320 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.748835 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.749677 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.749939 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.749999 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.751937 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.751938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.751939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.751945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.753479 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753825 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.753826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.753827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.753872 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.754043 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.754044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.754044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.754208 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.754267 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.754429 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.754491 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:01.754560 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.747777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:45121 (local address 127.0.0.1:52230)
0923 22:12:01.747810 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.747818 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.747848 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.748009 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.748010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.748153 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.748213 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.748214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.748929 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.748930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.749605 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.749606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.750007 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.750007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.750061 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.750470 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.750475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.751853 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.753566 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.753567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.753745 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.753941 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753941 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.753942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.753992 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.754277 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.754278 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.754306 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.754400 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.754481 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:12:01.754889 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.746549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.1.161.188:46283)
0923 22:12:01.746688 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.746691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.746705 (+ 14us) server_negotiation.cc:407] Connection header received
0923 22:12:01.746720 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.746721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.746730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.746801 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.747158 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.748063 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.749113 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.749440 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.749515 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.751368 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.751376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.751376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.751390 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.753139 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753638 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.753641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.753641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.753697 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.753897 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.753898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.753898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.754048 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.754125 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.754734 (+ 609us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.754804 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260923 22:12:01.754873 24061 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:01.756511 23986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: Bootstrap starting.
I20260923 22:12:01.757061 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.748145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.1.161.188:46283)
0923 22:12:01.748264 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.748266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.748350 (+ 84us) server_negotiation.cc:407] Connection header received
0923 22:12:01.748436 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.748437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.748445 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.748507 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.749190 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.749820 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.750847 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.751191 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.751253 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.753755 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.753756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.753756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.753762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.755938 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.756282 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.756283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.756283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.756336 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.756516 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.756516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.756517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.756684 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.756805 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.756878 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.756944 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260923 22:12:01.757419 23986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:01.757704 23986 log.cc:824] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: Log is configured to *not* fsync() on all Append() calls
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:01.759117 23986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: No bootstrap required, opened a new log
I20260923 22:12:01.761013 23986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.761145 23986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9bc6e29b4241659ca8412aa0cc16f0, State: Initialized, Role: FOLLOWER
I20260923 22:12:01.761252 23986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.761619 23986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: configured and running, proceeding with master startup.
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:12:01.762090 24068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.762176 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.755228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51546)
0923 22:12:01.755255 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.755263 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.755298 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.755538 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.755538 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.755696 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.755725 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.755727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.756362 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.756364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.757176 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.757179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.757611 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.757611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.757653 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.758005 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.758009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.759157 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.761242 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.761243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.761243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.761459 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.761649 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.761649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.761650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.761699 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.761913 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.761914 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.761946 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.762003 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.762077 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 22:12:01.762413 24068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.762460 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.755385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.1.161.188:46283)
0923 22:12:01.755437 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.755439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.755444 (+ 5us) server_negotiation.cc:407] Connection header received
0923 22:12:01.755458 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.755459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.755466 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.755510 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.755807 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.756297 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.757246 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.757505 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.757572 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.759234 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.759235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.759235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.759258 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.761156 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.761530 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.761531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.761531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.761575 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.761743 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.761744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.761744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.761838 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.761917 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.762304 (+ 387us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.762382 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:01.763636 24061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: Bootstrap starting.
I20260923 22:12:01.764264 24061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:01.764626 24061 log.cc:824] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:01.764900 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.757586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.1.161.190:39533)
0923 22:12:01.757614 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.757616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.757622 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:01.757637 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.757638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.757645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.757686 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.758215 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.758718 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.759596 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.759817 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.759876 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.762345 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.762346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.762347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.762352 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.763774 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.764129 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.764130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.764130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.764171 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.764355 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.764355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.764356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.764461 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.764557 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.764721 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.764789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:12:01.765321 24061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: No bootstrap required, opened a new log
I20260923 22:12:01.766489 24061 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.766579 24061 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95360a89f19464191a3ca4eec4d2822, State: Initialized, Role: FOLLOWER
I20260923 22:12:01.766672 24061 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.766889 24080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.766939 24061 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:01.766961 24080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:01.768888 24091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:01.768985 24091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:01.769131 23993 master_runner.cc:186] Error getting master registration for 127.1.161.190:39533: OK, instance_id { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" instance_seqno: 1790201521474090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:01.771354 23993 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:01.769107 23918 master_runner.cc:186] Error getting master registration for 127.1.161.190:39533: OK, instance_id { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" instance_seqno: 1790201521474090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:01.771423 23918 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 22:12:01.771788 24092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:01.771885 24092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:01.772240 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.765513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.1.161.189:45121)
0923 22:12:01.765549 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.765551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.765558 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:01.765603 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.765604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.765611 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.765655 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.766029 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.766395 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.767081 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.767317 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.767376 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.769654 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.769656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.769656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.769662 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.771202 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.771576 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.771577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.771577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.771621 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.771787 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.771788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.771788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.771906 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.772066 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.772108 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.772165 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:01.773213 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:01.774155 23885 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b7ea40ad8bad4418910a409b2f5ab941"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48160
I20260923 22:12:01.811197 23911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } attempt: 2
Sep 23 22:12:01 dist-test-slave-npvd krb5kdc[23830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:01.817499 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.811480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:46283 (local address 127.0.0.1:51562)
0923 22:12:01.811527 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:01.811536 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:01.811574 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:01.811727 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:01.811727 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:01.811889 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:01.811925 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.811927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.812703 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.812704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.813123 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:01.813124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.813501 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.813501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:01.813557 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.813917 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:01.813921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:01.815052 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:01.816541 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.816542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.816550 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.816752 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.816960 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:01.816961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:01.816961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:01.817013 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:01.817229 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:01.817230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:01.817268 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 22:12:01.817328 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 22:12:01.817419 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:01.817499 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.811533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.1.161.188:46283)
0923 22:12:01.811645 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.811646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.811653 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:01.811671 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.811671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.811679 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.811724 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.812000 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.812633 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.813196 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.813444 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.813501 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.815125 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.815127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.815127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.815133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.816462 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.816817 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.816824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.816824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.816877 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.817057 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.817058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.817058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.817179 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.817232 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.817355 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.817418 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 22:12:01.818755 23911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941: Bootstrap starting.
I20260923 22:12:01.819365 23911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:01.819619 23911 log.cc:824] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:01.820380 23911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941: No bootstrap required, opened a new log
I20260923 22:12:01.821674 23911 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.821789 23911 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ea40ad8bad4418910a409b2f5ab941, State: Initialized, Role: FOLLOWER
I20260923 22:12:01.821864 23911 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.822130 23911 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:01.822203 23885 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:12:01.822247 23885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:01.822988 23885 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.823226 23885 leader_election.cc:288] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9bc6e29b4241659ca8412aa0cc16f0 (127.1.161.189:45121), b95360a89f19464191a3ca4eec4d2822 (127.1.161.188:46283)
I20260923 22:12:01.823716 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7ea40ad8bad4418910a409b2f5ab941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0"
I20260923 22:12:01.823849 23960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:01.824620 23960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ea40ad8bad4418910a409b2f5ab941 in term 1.
I20260923 22:12:01.824816 23860 leader_election.cc:302] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9bc6e29b4241659ca8412aa0cc16f0, b7ea40ad8bad4418910a409b2f5ab941; no voters:
I20260923 22:12:01.825302 24035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7ea40ad8bad4418910a409b2f5ab941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b95360a89f19464191a3ca4eec4d2822"
I20260923 22:12:01.825408 24035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:01.826403 24035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ea40ad8bad4418910a409b2f5ab941 in term 1.
I20260923 22:12:01.822109 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.826746 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.826850 24096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:01.827028 24096 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [term 1 LEADER]: Becoming Leader. State: Replica: b7ea40ad8bad4418910a409b2f5ab941, State: Running, Role: LEADER
I20260923 22:12:01.827176 24096 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } }
I20260923 22:12:01.827591 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ea40ad8bad4418910a409b2f5ab941. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.827725 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:01.827795 23843 master_runner.cc:263] No leader master found from master b7ea40ad8bad4418910a409b2f5ab941
I20260923 22:12:01.827838 23843 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:01.828043 24114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:01.828732 24114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:01.836669 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.828863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.1.161.188:46283)
0923 22:12:01.828903 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.828904 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.830116 (+ 1212us) server_negotiation.cc:407] Connection header received
0923 22:12:01.830197 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.830198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.830211 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.830234 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.830929 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.831631 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.832451 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.832679 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.832711 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.834045 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.834046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.834046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.834052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.835521 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.835881 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.835882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.835883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.835928 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.836109 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.836110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.836110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.836223 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.836306 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.836488 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.836563 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:01.839170 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.828543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.1.161.190:39533)
0923 22:12:01.828587 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.828589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.829183 (+ 594us) server_negotiation.cc:407] Connection header received
0923 22:12:01.829265 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.829266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.829275 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.829298 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.831949 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.832517 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.834037 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.834288 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.834330 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.836559 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.836560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.836560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.836566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.838090 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.838436 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.838437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.838437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.838483 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.838687 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.838688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.838688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.838791 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.838855 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.839035 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.839099 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 22:12:01.842321 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:01.828320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.1.161.189:45121)
0923 22:12:01.828351 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.828353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.829403 (+ 1050us) server_negotiation.cc:407] Connection header received
0923 22:12:01.829458 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.829459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.829469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.829507 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.830159 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.830916 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.833326 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:01.833611 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:01.833652 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:01.839705 (+ 6053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:01.839706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:01.839707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:01.839713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:01.841235 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.841648 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.841649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.841649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.841695 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:01.841860 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:01.841861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:01.841861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:01.841964 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:01.842019 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:01.842186 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 22:12:01.842246 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:12:01.843252 24035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [term 1 FOLLOWER]: Refusing update from remote peer b7ea40ad8bad4418910a409b2f5ab941: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:01.843508 24096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:01.843868 23960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [term 1 FOLLOWER]: Refusing update from remote peer b7ea40ad8bad4418910a409b2f5ab941: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:01.844082 24096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:01.845048 24068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ea40ad8bad4418910a409b2f5ab941. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.845129 24068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.845809 24080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ea40ad8bad4418910a409b2f5ab941. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.845887 24080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.852052 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.852118 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:01.852504 24068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.852577 24068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.853780 24114 catalog_manager.cc:1347] Generated new cluster ID: 5b40dba71aa3464a90b80565b7cde776
I20260923 22:12:01.853770 24120 mvcc.cc:205] Tried to move back new op lower bound from 7332665433467768832 to 7332665433404923904. Current Snapshot: MvccSnapshot[applied={T|T < 7332665433467768832}]
I20260923 22:12:01.853820 24114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:01.863751 24112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.863842 24112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:01.864238 24080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ea40ad8bad4418910a409b2f5ab941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ea40ad8bad4418910a409b2f5ab941" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 39533 } } peers { permanent_uuid: "2f9bc6e29b4241659ca8412aa0cc16f0" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 45121 } } peers { permanent_uuid: "b95360a89f19464191a3ca4eec4d2822" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 46283 } } }
I20260923 22:12:01.864312 24080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:01.864749 24122 mvcc.cc:205] Tried to move back new op lower bound from 7332665433467768832 to 7332665433404923904. Current Snapshot: MvccSnapshot[applied={T|T < 7332665433467768832}]
I20260923 22:12:01.883410 24114 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:01.884037 24114 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:01.904126 24114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7ea40ad8bad4418910a409b2f5ab941: Generated new TSK 0
I20260923 22:12:01.904309 24114 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 22:12:01.929584 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.928123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.1.161.189:45121)
0923 22:12:01.928160 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.928162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.929003 (+ 841us) server_negotiation.cc:407] Connection header received
0923 22:12:01.929095 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.929095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.929105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.929133 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.929527 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52254: BlockingRecv error: recv got EOF from 127.0.0.1:52254 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 22:12:01.930358 23914 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.928348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.1.161.190:39533)
0923 22:12:01.928393 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.928395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.929924 (+ 1529us) server_negotiation.cc:407] Connection header received
0923 22:12:01.930007 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.930008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.930017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.930042 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.930308 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48184: BlockingRecv error: recv got EOF from 127.0.0.1:48184 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 22:12:01.931284 24064 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.928746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.1.161.188:46283)
0923 22:12:01.928783 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.928785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.930669 (+ 1884us) server_negotiation.cc:407] Connection header received
0923 22:12:01.930757 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.930758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.930767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.930794 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.931231 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51580: BlockingRecv error: recv got EOF from 127.0.0.1:51580 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 22:12:01.933274 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.932657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.1.161.189:45121)
0923 22:12:01.932686 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.932688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.932871 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:12:01.932955 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.932956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.932969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.932992 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.933238 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52268: BlockingRecv error: recv got EOF from 127.0.0.1:52268 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260923 22:12:01.936157 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.935505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.1.161.189:45121)
0923 22:12:01.935534 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.935536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.935734 (+ 198us) server_negotiation.cc:407] Connection header received
0923 22:12:01.935816 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.935817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.935826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.935849 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.936125 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52270: BlockingRecv error: recv got EOF from 127.0.0.1:52270 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260923 22:12:01.940025 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.939361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.1.161.189:45121)
0923 22:12:01.939402 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.939404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.939600 (+ 196us) server_negotiation.cc:407] Connection header received
0923 22:12:01.939683 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.939684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.939692 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.939717 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.939992 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52280: BlockingRecv error: recv got EOF from 127.0.0.1:52280 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260923 22:12:01.945688 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.945030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.1.161.189:45121)
0923 22:12:01.945056 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.945058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.945277 (+ 219us) server_negotiation.cc:407] Connection header received
0923 22:12:01.945356 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.945357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.945376 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.945401 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.945655 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52284: BlockingRecv error: recv got EOF from 127.0.0.1:52284 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260923 22:12:01.955304 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.954719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.1.161.189:45121)
0923 22:12:01.954778 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.954780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.954841 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:12:01.954895 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.954896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.954904 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.954939 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.955251 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52292: BlockingRecv error: recv got EOF from 127.0.0.1:52292 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 22:12:01.972749 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:01.972240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52298 (local address 127.1.161.189:45121)
0923 22:12:01.972274 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:01.972275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:01.972336 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:12:01.972409 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:01.972410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:01.972417 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:01.972470 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:01.972709 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52298: BlockingRecv error: recv got EOF from 127.0.0.1:52298 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:02.006471 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:02.005814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.1.161.189:45121)
0923 22:12:02.005869 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.005871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.005979 (+ 108us) server_negotiation.cc:407] Connection header received
0923 22:12:02.006080 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.006081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.006091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.006154 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:02.006408 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52306: BlockingRecv error: recv got EOF from 127.0.0.1:52306 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 22:12:02.072054 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:02.071492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.1.161.189:45121)
0923 22:12:02.071550 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.071552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.071634 (+ 82us) server_negotiation.cc:407] Connection header received
0923 22:12:02.071677 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.071678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.071686 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.071721 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:02.072003 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52310: BlockingRecv error: recv got EOF from 127.0.0.1:52310 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 22:12:02.201901 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:02.201242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.1.161.189:45121)
0923 22:12:02.201308 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.201311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.201361 (+ 50us) server_negotiation.cc:407] Connection header received
0923 22:12:02.201408 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.201409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.201418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.201481 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:02.201833 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52314: BlockingRecv error: recv got EOF from 127.0.0.1:52314 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260923 22:12:02.459678 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:02.459003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.1.161.189:45121)
0923 22:12:02.459079 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.459082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.459123 (+ 41us) server_negotiation.cc:407] Connection header received
0923 22:12:02.459215 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.459216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.459227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.459293 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:02.459594 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52318: BlockingRecv error: recv got EOF from 127.0.0.1:52318 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 22:12:02.769490 24091 catalog_manager.cc:1259] Loaded cluster ID: 5b40dba71aa3464a90b80565b7cde776
I20260923 22:12:02.769548 24091 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: loading cluster ID for follower catalog manager: success
I20260923 22:12:02.770264 24091 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: acquiring CA information for follower catalog manager: success
I20260923 22:12:02.770572 24091 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b95360a89f19464191a3ca4eec4d2822: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:02.772230 24092 catalog_manager.cc:1259] Loaded cluster ID: 5b40dba71aa3464a90b80565b7cde776
I20260923 22:12:02.772276 24092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: loading cluster ID for follower catalog manager: success
I20260923 22:12:02.772907 24092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: acquiring CA information for follower catalog manager: success
I20260923 22:12:02.773142 24092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f9bc6e29b4241659ca8412aa0cc16f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:02.975245 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:02.972721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.1.161.189:45121)
0923 22:12:02.972812 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.972813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.972823 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:02.972915 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.972916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.972974 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.973062 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:02.973207 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.973629 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.974451 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.974649 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.974725 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:02.974901 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:02.975003 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:02.975099 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:02.975120 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 22:12:02.975137 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":43,"threads_started":1}
I20260923 22:12:02.978802 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:02.976876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.1.161.188:46283)
0923 22:12:02.976983 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.976985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.976991 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:02.977022 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.977023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.977052 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.977107 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:02.977202 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.977604 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.978209 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.978363 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.978410 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:02.978529 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:02.978605 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:02.978650 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:02.978695 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 22:12:02.978708 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260923 22:12:02.983156 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:02.980183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.1.161.190:39533)
0923 22:12:02.980342 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.980344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.980353 (+ 9us) server_negotiation.cc:407] Connection header received
0923 22:12:02.980428 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.980429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.980481 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.980563 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:02.980737 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.981226 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982127 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.982316 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982398 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:02.982883 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:02.982965 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:02.983019 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:02.983058 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 22:12:02.983081 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":46,"threads_started":1}
I20260923 22:12:02.983209 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:02.980422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.1.161.188:46283)
0923 22:12:02.980457 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.980459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.980466 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:02.980538 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.980539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.980547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.980596 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:02.980768 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.981502 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982519 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.982761 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982814 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:02.982978 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:02.983002 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:02.983042 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:02.983124 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 22:12:02.983139 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:02.983338 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:02.980211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.1.161.189:45121)
0923 22:12:02.980270 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:02.980272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:02.980487 (+ 215us) server_negotiation.cc:407] Connection header received
0923 22:12:02.980601 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:02.980602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:02.980613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:02.980644 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:02.980858 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.981595 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982581 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:02.982822 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:02.982882 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:02.983024 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:02.983057 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:02.983085 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:02.983222 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:12:02.983244 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 22:12:02.984629 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23843
I20260923 22:12:02.989164 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23918
I20260923 22:12:02.993692 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 23993
2026-09-23T22:12:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2645 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 22:12:03 dist-test-slave-npvd krb5kdc[24210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 22:12:03 dist-test-slave-npvd krb5kdc[24210](info): set up 2 sockets
Sep 23 22:12:03 dist-test-slave-npvd krb5kdc[24210](info): commencing operation
krb5kdc: starting...
W20260923 22:12:03.923255 1670 mini_kdc.cc:121] Time spent starting KDC: real 0.911s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/test-user.keytab.
Sep 23 22:12:03 dist-test-slave-npvd krb5kdc[24210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T22:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T22:12:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.161.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
Entry for principal kudu/127.1.161.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab.
I20260923 22:12:04.024084 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33321
--webserver_interface=127.1.161.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 22:12:04.092006 24223 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:04.092160 24223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:04.092175 24223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:04.092191 24223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:04.093078 24223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:04.093106 24223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:04.093118 24223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:04.093129 24223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:04.093139 24223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:04.094277 24223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.190.keytab
--principal=kudu/127.1.161.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.190:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.161.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:04.094419 24223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:04.094545 24223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:04.098356 24229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:04.098400 24231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:04.098487 24223 server_base.cc:1034] running on GCE node
W20260923 22:12:04.098367 24228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:04.098815 24223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:04.099067 24223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:04.100219 24223 hybrid_clock.cc:648] HybridClock initialized: now 1790201524100204 us; error 26 us; skew 500 ppm
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:04.102412 24223 init.cc:377] Logged in from keytab as kudu/127.1.161.190@KRBTEST.COM (short username kudu)
I20260923 22:12:04.102782 24223 webserver.cc:460] Webserver started at http://127.1.161.190:42967/ using document root <none> and password file <none>
I20260923 22:12:04.102928 24223 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:04.102962 24223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:04.103030 24223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:04.103878 24223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data/instance:
uuid: "ab9010ce2fde4697aeed98f63fba8d71"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "7445a9ff8e498bac576a20a28f63ea5e"
server_key_iv: "5a272d6181f764d479398796941237d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.104115 24223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance:
uuid: "ab9010ce2fde4697aeed98f63fba8d71"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "7445a9ff8e498bac576a20a28f63ea5e"
server_key_iv: "5a272d6181f764d479398796941237d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.105108 24223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:04.105625 24238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:04.105767 24223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 22:12:04.105826 24223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal
uuid: "ab9010ce2fde4697aeed98f63fba8d71"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "7445a9ff8e498bac576a20a28f63ea5e"
server_key_iv: "5a272d6181f764d479398796941237d8"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.105876 24223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:04.115780 24223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:04.117223 24223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:04.117314 24223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:04.120786 24223 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.190:33321
I20260923 22:12:04.120847 24290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.190:33321 every 8 connection(s)
I20260923 22:12:04.121140 24223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/data/info.pb
I20260923 22:12:04.121968 24291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:04.123112 24291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.127568 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 24223
I20260923 22:12:04.127681 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-0/wal/instance
I20260923 22:12:04.127884 1670 external_mini_cluster.cc:1428] Setting key 5e6f83d5a463a1867d400a88a549c074
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:12:04.131525 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.123651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.1.161.190:33321)
0923 22:12:04.123789 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.123796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.123901 (+ 105us) server_negotiation.cc:407] Connection header received
0923 22:12:04.124322 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.124324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.124347 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.124438 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.124849 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.125577 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.126286 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.126554 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.126663 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.128793 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.128800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.128800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.128807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.130093 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.130483 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.130485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.130485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.130569 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.130803 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.130804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.130804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.130981 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.131037 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.131299 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.131395 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260923 22:12:04.131675 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.123626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33321 (local address 127.0.0.1:45978)
0923 22:12:04.123769 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.123801 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.124195 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.124441 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.124443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.124651 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.124764 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.124766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.125659 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.125660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.126209 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.126210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.126649 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.126650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.126702 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.127294 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.127302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.128721 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.130167 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.130173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.130175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.130415 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.130634 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.130635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.130635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.130740 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.131051 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.131058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.131138 (+ 80us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.131271 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.131491 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":69,"thread_start_us":70,"threads_started":1}
I20260923 22:12:04.132256 24291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.132516 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.132415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34508)
0923 22:12:04.132446 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.132489 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260923 22:12:04.132647 24242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
I20260923 22:12:04.133006 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.132908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34510)
0923 22:12:04.132952 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.132980 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:04.133124 24291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:41399: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.161.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
Entry for principal kudu/127.1.161.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab.
I20260923 22:12:04.149137 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:41399
--webserver_interface=127.1.161.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:04.182313 24291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } attempt: 1
I20260923 22:12:04.182693 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.182574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34524)
0923 22:12:04.182626 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.182663 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 22:12:04.183043 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.182950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34536)
0923 22:12:04.183000 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.183025 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 22:12:04.183156 24291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:41399: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
W20260923 22:12:04.218533 24298 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:04.218688 24298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:04.218703 24298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:04.218719 24298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:04.219687 24298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:04.219720 24298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:04.219732 24298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:04.219743 24298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:04.219774 24298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:04.221095 24298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.189.keytab
--principal=kudu/127.1.161.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.189:41399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.161.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:04.221315 24298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:04.221524 24298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 22:12:04.225553 24306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:04.225582 24304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:04.225582 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:04.225723 24298 server_base.cc:1034] running on GCE node
I20260923 22:12:04.226094 24298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:04.226329 24298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:04.227453 24298 hybrid_clock.cc:648] HybridClock initialized: now 1790201524227445 us; error 27 us; skew 500 ppm
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:04.234637 24298 init.cc:377] Logged in from keytab as kudu/127.1.161.189@KRBTEST.COM (short username kudu)
I20260923 22:12:04.235004 24298 webserver.cc:460] Webserver started at http://127.1.161.189:42743/ using document root <none> and password file <none>
I20260923 22:12:04.235147 24298 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:04.235234 24298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:04.235316 24298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:04.236080 24298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data/instance:
uuid: "d633ac8403a346fd98ac8c87c2537b94"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "c9a96ce69bd3ae780e6e3058c2bf4f6d"
server_key_iv: "cec174da9053a2df6dc70d6db1e075d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.236315 24298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal/instance:
uuid: "d633ac8403a346fd98ac8c87c2537b94"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "c9a96ce69bd3ae780e6e3058c2bf4f6d"
server_key_iv: "cec174da9053a2df6dc70d6db1e075d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.237337 24298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 22:12:04.237890 24313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:04.238062 24298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 22:12:04.238137 24298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal
uuid: "d633ac8403a346fd98ac8c87c2537b94"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "c9a96ce69bd3ae780e6e3058c2bf4f6d"
server_key_iv: "cec174da9053a2df6dc70d6db1e075d1"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.238198 24298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:04.251325 24291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } attempt: 2
I20260923 22:12:04.251658 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.251537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34550)
0923 22:12:04.251593 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.251627 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:12:04.251981 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.251914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34560)
0923 22:12:04.251948 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.251967 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 22:12:04.252080 24291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.189:41399: Network error: Client connection negotiation failed: client connection to 127.1.161.189:41399: connect: Connection refused (error 111)
I20260923 22:12:04.274533 24298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:04.276394 24298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:04.276526 24298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:04.279785 24298 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.189:41399
I20260923 22:12:04.279834 24365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.189:41399 every 8 connection(s)
I20260923 22:12:04.280118 24298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/data/info.pb
I20260923 22:12:04.281040 24366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:04.282407 24366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.283234 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 24298
I20260923 22:12:04.283362 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-1/wal/instance
I20260923 22:12:04.283532 1670 external_mini_cluster.cc:1428] Setting key e38346ccb1f9845224441a72e8956547
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.161.188@KRBTEST.COM; defaulting to no policy
I20260923 22:12:04.290342 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.282957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.1.161.190:33321)
0923 22:12:04.282998 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.283000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.283150 (+ 150us) server_negotiation.cc:407] Connection header received
0923 22:12:04.283651 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.283652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.283666 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.283706 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.284191 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.284606 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.285457 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.285639 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.285691 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.287770 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.287771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.287772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.287779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.289005 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.289435 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.289436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.289437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.289500 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.289742 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.289743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.289744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.289855 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.289939 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.290168 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.290249 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.290455 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.282915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33321 (local address 127.0.0.1:45992)
0923 22:12:04.283039 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.283073 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.283550 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.283757 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.283761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.283957 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.284109 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.284112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.284688 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.284689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.285367 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.285376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.285720 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.285721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.285851 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.286227 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.286235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.287655 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.289123 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.289129 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.289131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.289346 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.289573 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.289574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.289575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.289676 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.289919 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.289925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.290049 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.290150 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.290274 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260923 22:12:04.291006 24366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.161.188@KRBTEST.COM" created.
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:04.297870 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.291284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34564)
0923 22:12:04.291442 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.291454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.291500 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.291805 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.291806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.291904 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.291947 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.291948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.292654 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.292655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.293372 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.293374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.293941 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.293941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.293992 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.294302 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.294305 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.295305 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.297057 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.297058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.297058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.297194 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.297398 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.297399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.297399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.297441 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.297689 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.297690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.297717 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.297769 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.297822 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260923 22:12:04.297988 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.291483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.1.161.189:41399)
0923 22:12:04.291606 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.291610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.291628 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:12:04.291650 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.291656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.291666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.291723 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.292005 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.292575 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.293445 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.293857 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.293922 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.295397 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.295404 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.295404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.295411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.296965 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.297270 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.297273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.297273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.297333 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.297479 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.297479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.297479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.297631 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.297705 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.297799 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.297888 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260923 22:12:04.298442 24366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.298697 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.298608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36458)
0923 22:12:04.298646 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.298680 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 22:12:04.298803 24317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
I20260923 22:12:04.299127 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.299052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36468)
0923 22:12:04.299091 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.299110 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 22:12:04.299228 24366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:34641: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
Entry for principal kudu/127.1.161.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab.
I20260923 22:12:04.306255 1670 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9swwUu/build/release/bin/kudu
/tmp/dist-test-task9swwUu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:34641
--webserver_interface=127.1.161.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.161.148:37483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 22:12:04.343216 24291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } attempt: 3
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:04.348421 24366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } attempt: 1
I20260923 22:12:04.348749 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.348622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36478)
0923 22:12:04.348683 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.348715 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 22:12:04.349062 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.348966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36482)
0923 22:12:04.349011 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.349036 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 22:12:04.349148 24366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:34641: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
I20260923 22:12:04.349737 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.343553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.1.161.189:41399)
0923 22:12:04.343608 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.343609 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.343619 (+ 10us) server_negotiation.cc:407] Connection header received
0923 22:12:04.343673 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.343674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.343685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.343751 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.344025 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.344475 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.345199 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.345418 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.345490 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.347469 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.347471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.347471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.347477 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.348792 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.349144 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.349145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.349146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.349223 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.349398 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.349399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.349399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.349481 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.349524 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.349652 (+ 128us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.349697 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:04.349802 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.343500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34578)
0923 22:12:04.343555 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.343564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.343624 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.343756 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.343757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.343898 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.343952 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.343953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.344581 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.344582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.345114 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.345116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.345500 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.345501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.345574 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.345998 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.346003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.347382 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.348882 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.348883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.348883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.349072 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.349277 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.349278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.349278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.349338 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.349533 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.349534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.349576 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.349639 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.349722 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 22:12:04.350127 24291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.350414 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.350321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36492)
0923 22:12:04.350357 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.350391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:12:04.350742 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.350660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36504)
0923 22:12:04.350701 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.350726 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:04.350833 24291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:34641: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
W20260923 22:12:04.379065 24373 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 22:12:04.379279 24373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 22:12:04.379297 24373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 22:12:04.379333 24373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 22:12:04.380322 24373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 22:12:04.380358 24373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 22:12:04.380370 24373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 22:12:04.380400 24373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 22:12:04.380421 24373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 22:12:04.381834 24373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.161.148:37483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.161.190:33321,127.1.161.189:41399,127.1.161.188:34641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/krb5kdc/kudu_127.1.161.188.keytab
--principal=kudu/127.1.161.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.161.188:34641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.161.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 20:53:37 UTC on ccycloud.quasar-rkviyx.root.comops.site
build id 4668083
I20260923 22:12:04.382043 24373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 22:12:04.381991 24291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } attempt: 1
I20260923 22:12:04.382277 24373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 22:12:04.382354 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.382226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36508)
0923 22:12:04.382280 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.382323 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 22:12:04.382658 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.382575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36514)
0923 22:12:04.382615 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.382638 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 22:12:04.382761 24291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:34641: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
W20260923 22:12:04.386291 24378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 22:12:04.386312 24379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:04.386431 24373 server_base.cc:1034] running on GCE node
W20260923 22:12:04.386300 24381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 22:12:04.386848 24373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 22:12:04.387091 24373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 22:12:04.388218 24373 hybrid_clock.cc:648] HybridClock initialized: now 1790201524388207 us; error 23 us; skew 500 ppm
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 22:12:04.390245 24373 init.cc:377] Logged in from keytab as kudu/127.1.161.188@KRBTEST.COM (short username kudu)
I20260923 22:12:04.390666 24373 webserver.cc:460] Webserver started at http://127.1.161.188:41523/ using document root <none> and password file <none>
I20260923 22:12:04.390884 24373 fs_manager.cc:359] Metadata directory not provided
I20260923 22:12:04.390928 24373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 22:12:04.391075 24373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 22:12:04.392076 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data/instance:
uuid: "daef0cab1da049e69860de17ab76fe23"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "04e81488b7efbddf900e1f1c5a808b3a"
server_key_iv: "bf9976af7285c042020df0c20e2b8f5a"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.392409 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal/instance:
uuid: "daef0cab1da049e69860de17ab76fe23"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "04e81488b7efbddf900e1f1c5a808b3a"
server_key_iv: "bf9976af7285c042020df0c20e2b8f5a"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.393509 24373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 22:12:04.394253 24388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 22:12:04.394455 24373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 22:12:04.394539 24373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data,/tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal
uuid: "daef0cab1da049e69860de17ab76fe23"
format_stamp: "Formatted at 2026-09-23 22:12:04 on dist-test-slave-npvd"
server_key: "04e81488b7efbddf900e1f1c5a808b3a"
server_key_iv: "bf9976af7285c042020df0c20e2b8f5a"
server_key_version: "encryptionkey@0"
I20260923 22:12:04.394603 24373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 22:12:04.412550 24373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 22:12:04.413960 24373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 22:12:04.414093 24373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 22:12:04.417265 24366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } attempt: 2
I20260923 22:12:04.417629 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.417514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36522)
0923 22:12:04.417566 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.417596 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 22:12:04.417996 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.417879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36530)
0923 22:12:04.417943 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.417973 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 22:12:04.418089 24366 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.161.188:34641: Network error: Client connection negotiation failed: client connection to 127.1.161.188:34641: connect: Connection refused (error 111)
I20260923 22:12:04.418182 24373 rpc_server.cc:307] RPC server started. Bound to: 127.1.161.188:34641
I20260923 22:12:04.418223 24440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.161.188:34641 every 8 connection(s)
I20260923 22:12:04.418534 24373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/data/info.pb
I20260923 22:12:04.419529 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 22:12:04.420037 1670 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9swwUu/build/release/bin/kudu as pid 24373
I20260923 22:12:04.420156 1670 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9swwUu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790201274647240-1670-0/minicluster-data/master-2/wal/instance
I20260923 22:12:04.420399 1670 external_mini_cluster.cc:1428] Setting key 2ec23ea29dc597f5ba24353670aaa110
I20260923 22:12:04.421257 24441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:04.429769 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.422063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.1.161.190:33321)
0923 22:12:04.422125 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.422127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.422294 (+ 167us) server_negotiation.cc:407] Connection header received
0923 22:12:04.422325 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.422326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.422342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.422400 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.422907 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.423585 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.424697 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.424950 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.425008 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.426951 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.426952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.426952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.426959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.428549 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.428882 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.428884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.428884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.428937 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.429141 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.429142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.429142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.429258 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.429302 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.429621 (+ 319us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.429690 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 22:12:04.429972 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.422020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.190:33321 (local address 127.0.0.1:45994)
0923 22:12:04.422163 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.422207 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.422281 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.422412 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.422414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.422744 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.422826 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.422830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.423675 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.423676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.424633 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.424634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.425045 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.425046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.425117 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.425462 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.425471 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.426873 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.428623 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.428628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.428629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.428813 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.429000 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.429001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.429001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.429082 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.429315 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.429321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.429465 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.429579 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.429712 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260923 22:12:04.430454 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.420865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.1.161.188:34641)
0923 22:12:04.421025 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.421031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.421054 (+ 23us) server_negotiation.cc:407] Connection header received
0923 22:12:04.421419 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.421424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.421471 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.421573 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.422328 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.423477 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.424502 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.424831 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.424926 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.427469 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.427475 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.427476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.427482 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.429088 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.429408 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.429409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.429410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.429454 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.429635 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.429636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.429636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.429774 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.429920 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.430284 (+ 364us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.430354 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260923 22:12:04.430474 24441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.190@KRBTEST.COM
I20260923 22:12:04.436704 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.430679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.189:41399 (local address 127.0.0.1:34590)
0923 22:12:04.430713 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.430722 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.430754 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.431065 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.431066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.431237 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.431296 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.431298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.432001 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.432001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.432526 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.432529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.432833 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.432834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.432896 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.433222 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.433225 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.434364 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.435819 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.435819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.435820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.436008 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.436183 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.436184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.436184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.436250 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.436492 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.436493 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.436521 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.436577 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.436654 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 22:12:04.436774 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.430887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.1.161.189:41399)
0923 22:12:04.430942 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.430943 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.430951 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:04.430972 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.430972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.430987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.431047 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.431373 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.431931 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.432596 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.432775 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.432818 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.434448 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.434449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.434449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.434456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.435746 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.436073 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.436074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.436074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.436121 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.436304 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.436305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.436305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.436429 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.436493 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.436633 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.436701 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 22:12:04.437287 24441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } has no permanent_uuid. Determining permanent_uuid...
I20260923 22:12:04.438205 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.431141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.1.161.190:33321)
0923 22:12:04.431192 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.431193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.431257 (+ 64us) server_negotiation.cc:407] Connection header received
0923 22:12:04.431320 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.431321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.431327 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.431404 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.431802 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.432250 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.432978 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.433232 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.433303 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.435579 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.435580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.435580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.435587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.437067 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.437471 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.437472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.437473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.437521 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.437721 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.437722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.437722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.437832 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.438000 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.438038 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.438114 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.188@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.161.189@KRBTEST.COM
I20260923 22:12:04.443430 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.437528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36548)
0923 22:12:04.437564 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.437572 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.437607 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.437723 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.437724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.437850 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.437869 (+ 19us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.437870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.438547 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.438548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.439329 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.439331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.439634 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.439634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.439673 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.439978 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.439982 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.441047 (+ 1065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.442492 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.442493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.442493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.442633 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.442816 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.442817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.442817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.442863 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.443088 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.443089 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.443120 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.443238 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.443324 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 22:12:04.443490 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.437565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.1.161.188:34641)
0923 22:12:04.437597 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.437599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.437607 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:04.437661 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.437661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.437670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.437711 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.437941 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.438481 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.439393 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.439574 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.439636 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.441126 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.441127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.441127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.441134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.442406 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.442706 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.442707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.442707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.442749 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.442929 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.442929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.442930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.443023 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.443085 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.443261 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.443387 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":18}
I20260923 22:12:04.444723 24441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: Bootstrap starting.
I20260923 22:12:04.445432 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.438692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.1.161.189:41399)
0923 22:12:04.438727 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.438729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.438747 (+ 18us) server_negotiation.cc:407] Connection header received
0923 22:12:04.438760 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.438761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.438767 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.438800 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.439270 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.439609 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.440156 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.440401 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.440480 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.442754 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.442755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.442755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.442761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.444431 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.444692 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.444693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.444693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.444742 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.444920 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.444921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.444921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.445038 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.445198 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.445233 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.445368 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:04.445566 24441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:04.445824 24441 log.cc:824] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:04.446498 24441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: No bootstrap required, opened a new log
I20260923 22:12:04.446597 1670 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 22:12:04.447810 24441 raft_consensus.cc:348] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.447933 24441 raft_consensus.cc:729] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daef0cab1da049e69860de17ab76fe23, State: Initialized, Role: FOLLOWER
I20260923 22:12:04.448012 24441 consensus_queue.cc:260] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.447433 24265 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ab9010ce2fde4697aeed98f63fba8d71"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998
I20260923 22:12:04.448249 24447 sys_catalog.cc:455] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.448307 24441 sys_catalog.cc:564] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:04.448325 24447 sys_catalog.cc:458] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 22:12:04.449980 24458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:04.450035 24458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:04.450227 24373 master_runner.cc:186] Error getting master registration for 127.1.161.190:33321: OK, instance_id { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" instance_seqno: 1790201524119816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:04.452178 24373 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:04.457898 24291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } attempt: 2
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.190@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:04.464510 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.458143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36558)
0923 22:12:04.458180 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.458189 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.458223 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.458345 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.458346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.458507 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.458543 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.458544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.459320 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.459321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.460011 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.460013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.460439 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.460441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.460490 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.460928 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.460933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.462052 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.463609 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.463610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.463611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.463788 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.463961 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.463962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.463962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.464012 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.464260 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.464261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.464295 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.464354 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.464427 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 22:12:04.464525 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.458196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36558 (local address 127.1.161.188:34641)
0923 22:12:04.458225 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.458227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.458257 (+ 30us) server_negotiation.cc:407] Connection header received
0923 22:12:04.458273 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.458274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.458281 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.458325 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.458626 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.459256 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.460100 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.460361 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.460435 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.462128 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.462129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.462130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.462136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.463539 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.463854 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.463855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.463855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.463904 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.464066 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.464066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.464066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.464195 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.464253 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.464391 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.464453 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.465777 24291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: Bootstrap starting.
I20260923 22:12:04.466409 24291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:04.466681 24291 log.cc:824] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:04.467538 24291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: No bootstrap required, opened a new log
I20260923 22:12:04.468806 24291 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.468927 24291 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9010ce2fde4697aeed98f63fba8d71, State: Initialized, Role: FOLLOWER
I20260923 22:12:04.469002 24291 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.469264 24291 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:04.469285 24265 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 22:12:04.469239 24461 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.469379 24461 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.469338 24265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:04.470072 24265 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.470310 24265 leader_election.cc:288] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [CANDIDATE]: Term 1 election: Requested vote from peers d633ac8403a346fd98ac8c87c2537b94 (127.1.161.189:41399), daef0cab1da049e69860de17ab76fe23 (127.1.161.188:34641)
I20260923 22:12:04.470891 24415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab9010ce2fde4697aeed98f63fba8d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "daef0cab1da049e69860de17ab76fe23"
I20260923 22:12:04.471006 24415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [term 0 FOLLOWER]: Advancing to term 1
W20260923 22:12:04.471830 24472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 22:12:04.471908 24472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 22:12:04.472070 24223 master_runner.cc:186] Error getting master registration for 127.1.161.189:41399: OK, instance_id { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" instance_seqno: 1790201524278841 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 22:12:04.474346 24223 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 22:12:04.475036 24415 raft_consensus.cc:2454] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab9010ce2fde4697aeed98f63fba8d71 in term 1.
I20260923 22:12:04.475251 24242 leader_election.cc:302] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab9010ce2fde4697aeed98f63fba8d71, daef0cab1da049e69860de17ab76fe23; no voters:
I20260923 22:12:04.475356 24461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 22:12:04.475490 24461 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [term 1 LEADER]: Becoming Leader. State: Replica: ab9010ce2fde4697aeed98f63fba8d71, State: Running, Role: LEADER
I20260923 22:12:04.475564 24461 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.475927 24480 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9010ce2fde4697aeed98f63fba8d71. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.476373 24480 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:04.476603 24483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 22:12:04.476815 24483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 22:12:04.478446 24415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [term 1 FOLLOWER]: Refusing update from remote peer ab9010ce2fde4697aeed98f63fba8d71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:04.478457 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.471430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.1.161.189:41399)
0923 22:12:04.471468 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.471470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.472048 (+ 578us) server_negotiation.cc:407] Connection header received
0923 22:12:04.472154 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.472155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.472164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.472213 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.472775 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.473175 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.474020 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.474302 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.474362 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.475757 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.475758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.475758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.475763 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.477246 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.477631 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.477632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.477633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.477687 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.477916 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.477917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.477918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.478036 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.478100 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.478292 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.478373 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 22:12:04.479267 24461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 22:12:04.480459 24447 sys_catalog.cc:455] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9010ce2fde4697aeed98f63fba8d71. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.480561 24447 sys_catalog.cc:458] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.484222 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.476178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.1.161.190:33321)
0923 22:12:04.476207 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.476208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.477588 (+ 1380us) server_negotiation.cc:407] Connection header received
0923 22:12:04.477678 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.477679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.477688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.477727 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.478418 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.478986 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.479887 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.480092 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.480119 (+ 27us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.481520 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.481521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.481522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.481528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.482998 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.483407 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.483408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.483408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.483457 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.483677 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.483678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.483678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.483808 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.483899 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.484069 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.484132 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:12:04.485383 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.471964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.1.161.188:34641)
0923 22:12:04.475931 (+ 3967us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.475932 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.475940 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:04.475956 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.475956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.475964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.476024 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.479318 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.479899 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.481107 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.481414 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.481455 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.482876 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.482877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.482878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.482883 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.484309 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.484664 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.484665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.484665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.484722 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.484917 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.484918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.484918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.485036 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.485108 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.485240 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.485301 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3951}
I20260923 22:12:04.492148 24480 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.492245 24480 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:04.492312 24447 sys_catalog.cc:455] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.492386 24447 sys_catalog.cc:458] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.492491 24484 mvcc.cc:205] Tried to move back new op lower bound from 7332665444261134336 to 7332665444252446720. Current Snapshot: MvccSnapshot[applied={T|T < 7332665444261134336}]
I20260923 22:12:04.492578 24485 mvcc.cc:205] Tried to move back new op lower bound from 7332665444261134336 to 7332665444252446720. Current Snapshot: MvccSnapshot[applied={T|T < 7332665444261134336}]
I20260923 22:12:04.493255 24483 catalog_manager.cc:1347] Generated new cluster ID: bf4848b32aaa4a7da4fb4d9004b61690
I20260923 22:12:04.493295 24483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 22:12:04.507148 24483 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 22:12:04.507980 24483 catalog_manager.cc:1504] Loading token signing keys...
I20260923 22:12:04.509217 24366 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } attempt: 3
Sep 23 22:12:04 dist-test-slave-npvd krb5kdc[24210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.161.189@KRBTEST.COM for kudu/127.1.161.188@KRBTEST.COM
I20260923 22:12:04.516801 24483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71: Generated new TSK 0
I20260923 22:12:04.516897 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.509422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.161.188:34641 (local address 127.0.0.1:36568)
0923 22:12:04.509484 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 22:12:04.509494 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 22:12:04.509524 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 22:12:04.509782 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 22:12:04.509783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 22:12:04.509952 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 22:12:04.509988 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.509989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.510775 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.510776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.511563 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 22:12:04.511566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.512018 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.512018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 22:12:04.512061 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.512483 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 22:12:04.512488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 22:12:04.514093 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 22:12:04.515800 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.515800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.515801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.516003 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.516228 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 22:12:04.516229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 22:12:04.516229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 22:12:04.516288 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 22:12:04.516588 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 22:12:04.516588 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 22:12:04.516646 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 22:12:04.516710 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 22:12:04.516800 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 22:12:04.516948 24483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 22:12:04.517454 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.509595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.1.161.188:34641)
0923 22:12:04.509672 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.509673 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.509679 (+ 6us) server_negotiation.cc:407] Connection header received
0923 22:12:04.509708 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.509708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.509716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.509751 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.510047 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.510703 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.511652 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.511937 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.511979 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.514178 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 22:12:04.514179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 22:12:04.514179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 22:12:04.514184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 22:12:04.515725 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.516103 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.516104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.516104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.516169 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 22:12:04.516369 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 22:12:04.516370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 22:12:04.516371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 22:12:04.516500 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 22:12:04.516565 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.517282 (+ 717us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.517365 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 22:12:04.518533 24366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: Bootstrap starting.
I20260923 22:12:04.519155 24366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: Neither blocks nor log segments found. Creating new log.
I20260923 22:12:04.519464 24366 log.cc:824] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: Log is configured to *not* fsync() on all Append() calls
I20260923 22:12:04.520139 24366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: No bootstrap required, opened a new log
I20260923 22:12:04.521575 24366 raft_consensus.cc:348] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.521703 24366 raft_consensus.cc:729] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d633ac8403a346fd98ac8c87c2537b94, State: Initialized, Role: FOLLOWER
I20260923 22:12:04.521807 24366 consensus_queue.cc:260] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } }
I20260923 22:12:04.522116 24490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.522220 24490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.522498 24340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab9010ce2fde4697aeed98f63fba8d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d633ac8403a346fd98ac8c87c2537b94"
I20260923 22:12:04.522588 24340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [term 0 FOLLOWER]: Advancing to term 1
I20260923 22:12:04.523612 24340 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab9010ce2fde4697aeed98f63fba8d71 in term 1.
I20260923 22:12:04.523927 24339 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [term 1 FOLLOWER]: Refusing update from remote peer ab9010ce2fde4697aeed98f63fba8d71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 22:12:04.524135 24461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 22:12:04.527019 24319 tablet.cc:2369] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 22:12:04.522181 24366 sys_catalog.cc:564] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 22:12:04.527593 24490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9010ce2fde4697aeed98f63fba8d71. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.527665 24490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.527702 24490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.527752 24490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 22:12:04.529536 24492 mvcc.cc:205] Tried to move back new op lower bound from 7332665444415365120 to 7332665444252446720. Current Snapshot: MvccSnapshot[applied={T|T < 7332665444261134336}]
I20260923 22:12:04.530833 24505 catalog_manager.cc:1259] Loaded cluster ID: bf4848b32aaa4a7da4fb4d9004b61690
I20260923 22:12:04.530925 24505 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: loading cluster ID for follower catalog manager: success
I20260923 22:12:04.532397 24505 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: acquiring CA information for follower catalog manager: success
I20260923 22:12:04.532701 24505 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d633ac8403a346fd98ac8c87c2537b94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 22:12:04.545504 24370 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.544103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.1.161.189:41399)
0923 22:12:04.544137 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.544139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.544954 (+ 815us) server_negotiation.cc:407] Connection header received
0923 22:12:04.545056 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.545057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.545066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.545092 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.545458 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34628: BlockingRecv error: recv got EOF from 127.0.0.1:34628 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:12:04.546305 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.544389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.1.161.188:34641)
0923 22:12:04.544419 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.544420 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.545708 (+ 1288us) server_negotiation.cc:407] Connection header received
0923 22:12:04.545952 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.545953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.545962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.545990 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.546255 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36578: BlockingRecv error: recv got EOF from 127.0.0.1:36578 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:12:04.546833 24294 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.544695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.1.161.190:33321)
0923 22:12:04.544728 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.544730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.545806 (+ 1076us) server_negotiation.cc:407] Connection header received
0923 22:12:04.546439 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.546440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.546450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.546479 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.546780 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46028: BlockingRecv error: recv got EOF from 127.0.0.1:46028 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 22:12:04.551578 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.548287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.1.161.189:41399)
0923 22:12:04.548314 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.548316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.548515 (+ 199us) server_negotiation.cc:407] Connection header received
0923 22:12:04.548602 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.548603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.548628 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.548664 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.548833 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.549541 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.550639 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.550942 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.550981 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.551228 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.551343 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.551392 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.551504 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.551520 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260923 22:12:04.554139 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.553495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.1.161.188:34641)
0923 22:12:04.553534 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.553536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.553723 (+ 187us) server_negotiation.cc:407] Connection header received
0923 22:12:04.553806 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.553807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.553819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.553855 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.554103 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36590: BlockingRecv error: recv got EOF from 127.0.0.1:36590 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 22:12:04.558995 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.556057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34646 (local address 127.1.161.189:41399)
0923 22:12:04.556085 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.556087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.556282 (+ 195us) server_negotiation.cc:407] Connection header received
0923 22:12:04.556363 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.556364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.556371 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.556395 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.556536 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.557264 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.558312 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.558569 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.558605 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.558769 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.558797 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.558820 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.558925 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.558940 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:12:04.561657 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.560983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.1.161.188:34641)
0923 22:12:04.561015 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.561016 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.561220 (+ 204us) server_negotiation.cc:407] Connection header received
0923 22:12:04.561307 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.561308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.561317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.561345 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.561614 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36598: BlockingRecv error: recv got EOF from 127.0.0.1:36598 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.567492 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.564549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.1.161.189:41399)
0923 22:12:04.564580 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.564582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.564787 (+ 205us) server_negotiation.cc:407] Connection header received
0923 22:12:04.564886 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.564887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.564894 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.564920 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.565063 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.565728 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.566747 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.567035 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.567071 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.567273 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.567300 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.567325 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.567420 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.567434 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:12:04.570164 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.569509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.1.161.188:34641)
0923 22:12:04.569539 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.569540 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.569741 (+ 201us) server_negotiation.cc:407] Connection header received
0923 22:12:04.569819 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.569820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.569828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.569865 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.570126 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36612: BlockingRecv error: recv got EOF from 127.0.0.1:36612 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 22:12:04.578097 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.575060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.1.161.189:41399)
0923 22:12:04.575090 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.575092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.575340 (+ 248us) server_negotiation.cc:407] Connection header received
0923 22:12:04.575427 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.575428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.575435 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.575460 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.575599 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.576283 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.577361 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.577646 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.577683 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.577889 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.577915 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.577940 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.578033 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.578046 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:12:04.580586 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.579944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.1.161.188:34641)
0923 22:12:04.579975 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.579977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.580155 (+ 178us) server_negotiation.cc:407] Connection header received
0923 22:12:04.580235 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.580236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.580244 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.580269 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.580518 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36618: BlockingRecv error: recv got EOF from 127.0.0.1:36618 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.592320 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.589460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.1.161.189:41399)
0923 22:12:04.589488 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.589490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.589674 (+ 184us) server_negotiation.cc:407] Connection header received
0923 22:12:04.589748 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.589749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.589755 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.589777 (+ 22us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.589922 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.590529 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.591648 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.591935 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.591968 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.592124 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.592148 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.592170 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.592253 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.592266 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 22:12:04.594784 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.594157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.1.161.188:34641)
0923 22:12:04.594190 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.594191 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.594374 (+ 183us) server_negotiation.cc:407] Connection header received
0923 22:12:04.594453 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.594454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.594463 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.594489 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.594737 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36620: BlockingRecv error: recv got EOF from 127.0.0.1:36620 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.614769 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.612008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.1.161.189:41399)
0923 22:12:04.612039 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.612040 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.612266 (+ 226us) server_negotiation.cc:407] Connection header received
0923 22:12:04.612353 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.612354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.612362 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.612387 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.612545 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.613188 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.614125 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.614350 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.614390 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.614562 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.614589 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.614612 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.614702 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.614716 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 22:12:04.617512 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.616645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.1.161.188:34641)
0923 22:12:04.616676 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.616678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.617002 (+ 324us) server_negotiation.cc:407] Connection header received
0923 22:12:04.617103 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.617104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.617114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.617143 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.617471 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36636: BlockingRecv error: recv got EOF from 127.0.0.1:36636 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 22:12:04.653740 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.650736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.1.161.189:41399)
0923 22:12:04.650780 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.650782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.650921 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:12:04.650997 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.650998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.651009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.651086 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.651249 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.651997 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.652930 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.653229 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.653305 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.653478 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.653513 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.653567 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.653639 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.653656 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 22:12:04.655961 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.655402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.1.161.188:34641)
0923 22:12:04.655447 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.655449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.655511 (+ 62us) server_negotiation.cc:407] Connection header received
0923 22:12:04.655572 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.655572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.655580 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.655620 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.655914 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36646: BlockingRecv error: recv got EOF from 127.0.0.1:36646 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 22:12:04.723053 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.720904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.1.161.189:41399)
0923 22:12:04.720969 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.720971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.721040 (+ 69us) server_negotiation.cc:407] Connection header received
0923 22:12:04.721133 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.721134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.721145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.721200 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.721337 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.721821 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.722470 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.722640 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.722705 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.722824 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.722855 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.722898 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.722948 (+ 50us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.722964 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260923 22:12:04.725150 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.724568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.1.161.188:34641)
0923 22:12:04.724641 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.724643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.724708 (+ 65us) server_negotiation.cc:407] Connection header received
0923 22:12:04.724770 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.724771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.724782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.724822 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.725100 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36654: BlockingRecv error: recv got EOF from 127.0.0.1:36654 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 22:12:04.855989 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:04.854154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.1.161.189:41399)
0923 22:12:04.854223 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.854225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.854276 (+ 51us) server_negotiation.cc:407] Connection header received
0923 22:12:04.854330 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.854331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.854339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.854384 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:04.854485 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.854878 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.855518 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:04.855666 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:04.855707 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:04.855808 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:04.855829 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:04.855860 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:04.855926 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 22:12:04.855939 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 22:12:04.858620 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:04.857552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.1.161.188:34641)
0923 22:12:04.857746 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:04.857749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:04.857888 (+ 139us) server_negotiation.cc:407] Connection header received
0923 22:12:04.858017 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:04.858018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:04.858031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:04.858081 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:04.858537 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36668: BlockingRecv error: recv got EOF from 127.0.0.1:36668 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
I20260923 22:12:04.975436 24461 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab9010ce2fde4697aeed98f63fba8d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab9010ce2fde4697aeed98f63fba8d71" member_type: VOTER last_known_addr { host: "127.1.161.190" port: 33321 } } peers { permanent_uuid: "d633ac8403a346fd98ac8c87c2537b94" member_type: VOTER last_known_addr { host: "127.1.161.189" port: 41399 } } peers { permanent_uuid: "daef0cab1da049e69860de17ab76fe23" member_type: VOTER last_known_addr { host: "127.1.161.188" port: 34641 } } }
I20260923 22:12:04.975602 24461 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9010ce2fde4697aeed98f63fba8d71 [sys.catalog]: This master's current role is: LEADER
I20260923 22:12:05.117478 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.115631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.1.161.189:41399)
0923 22:12:05.115670 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.115674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.115701 (+ 27us) server_negotiation.cc:407] Connection header received
0923 22:12:05.115760 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.115761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.115768 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.115802 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.115927 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.116324 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.116981 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.117136 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.117179 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.117300 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.117322 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.117355 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.117412 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.117426 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 22:12:05.119386 24444 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 22:12:05.118908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.1.161.188:34641)
0923 22:12:05.118969 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.118971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.118978 (+ 7us) server_negotiation.cc:407] Connection header received
0923 22:12:05.118999 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.119000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.119019 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.119058 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 22:12:05.119334 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36672: BlockingRecv error: recv got EOF from 127.0.0.1:36672 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 22:12:05.450387 24458 catalog_manager.cc:1259] Loaded cluster ID: bf4848b32aaa4a7da4fb4d9004b61690
I20260923 22:12:05.450469 24458 catalog_manager.cc:1552] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: loading cluster ID for follower catalog manager: success
I20260923 22:12:05.451294 24458 catalog_manager.cc:1574] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: acquiring CA information for follower catalog manager: success
I20260923 22:12:05.451596 24458 catalog_manager.cc:1602] T 00000000000000000000000000000000 P daef0cab1da049e69860de17ab76fe23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 22:12:05.634542 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.632418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.1.161.189:41399)
0923 22:12:05.632511 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.632513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.632524 (+ 11us) server_negotiation.cc:407] Connection header received
0923 22:12:05.632596 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.632597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.632607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.632677 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.632805 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.633230 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.633887 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.634094 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.634161 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.634295 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.634330 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.634389 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.634435 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.634453 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":45,"threads_started":1}
I20260923 22:12:05.638118 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.636034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.1.161.188:34641)
0923 22:12:05.636115 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.636117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.636125 (+ 8us) server_negotiation.cc:407] Connection header received
0923 22:12:05.636180 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.636181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.636244 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.636314 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.636395 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.636807 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.637454 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.637649 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.637714 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.637833 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.637931 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.638011 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.638028 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.638044 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"thread_start_us":37,"threads_started":1}
I20260923 22:12:05.642196 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.639656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.1.161.188:34641)
0923 22:12:05.639687 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.639688 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.639786 (+ 98us) server_negotiation.cc:407] Connection header received
0923 22:12:05.639835 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.639836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.639851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.639893 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.640034 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.640477 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.641398 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.641668 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.641740 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.641955 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.641994 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.642044 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.642109 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.642128 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 22:12:05.642541 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.639483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.1.161.189:41399)
0923 22:12:05.639520 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.639522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.639636 (+ 114us) server_negotiation.cc:407] Connection header received
0923 22:12:05.639722 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.639723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.639730 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.639752 (+ 22us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.639892 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.640509 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.641783 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.642126 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.642177 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.642343 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.642367 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.642389 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.642463 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.642479 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 22:12:05.642928 24636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 22:12:05.639466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.1.161.190:33321)
0923 22:12:05.639555 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 22:12:05.639557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 22:12:05.639618 (+ 61us) server_negotiation.cc:407] Connection header received
0923 22:12:05.639737 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 22:12:05.639739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 22:12:05.639782 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 22:12:05.639850 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 22:12:05.640034 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.640736 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.641888 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 22:12:05.642166 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 22:12:05.642222 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 22:12:05.642559 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 22:12:05.642651 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 22:12:05.642688 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 22:12:05.642825 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 22:12:05.642849 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":51,"threads_started":1}
I20260923 22:12:05.644539 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 24223
I20260923 22:12:05.649569 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 24298
I20260923 22:12:05.654080 1670 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9swwUu/build/release/bin/kudu with pid 24373
2026-09-23T22:12:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2660 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5305 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (251014 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST